You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/09/07 02:31:27 UTC
[GitHub] [iotdb] zyk990424 opened a new pull request #3918: Wildcard extension
zyk990424 opened a new pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918
## Description
### Content1 ...
### Content2 ...
### Content3 ...
<!--
In each section, please describe design decisions made, including:
- Choice of algorithms
- Behavioral aspects. What configuration values are acceptable? How are corner cases and error
conditions handled, such as when there are insufficient resources?
- Class organization and design (how the logic is split between classes, inheritance, composition,
design patterns)
- Method organization and design (how the logic is split between methods, parameters and return types)
- Naming (class, method, API, configuration, HTTP endpoint, names of emitted metrics)
-->
<!-- It's good to describe an alternative design (or mention an alternative name) for every design
(or naming) decision point and compare the alternatives with the designs that you've implemented
(or the names you've chosen) to highlight the advantages of the chosen designs and names. -->
<!-- If there was a discussion of the design of the feature implemented in this PR elsewhere
(e. g. a "Proposal" issue, any other issue, or a thread in the development mailing list),
link to that discussion from this PR description and explain what have changed in your final design
compared to your original proposal or the consensus version in the end of the discussion.
If something hasn't changed since the original discussion, you can omit a detailed discussion of
those aspects of the design here, perhaps apart from brief mentioning for the sake of readability
of this PR description. -->
<!-- Some of the aspects mentioned above may be omitted for simple and small changes. -->
<hr>
This PR has:
- [ ] been self-reviewed.
- [ ] concurrent read
- [ ] concurrent write
- [ ] concurrent read and write
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods.
- [ ] added or updated version, __license__, or notice information
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious
for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold
for code coverage.
- [ ] added integration tests.
- [ ] been tested in a test IoTDB cluster.
<!-- Check the items by putting "x" in the brackets for the done things. Not all of these items
apply to every PR. Remove the items which are not done or not relevant to the PR. None of the items
from the checklist above are strictly necessary, but it would be very helpful if you at least
self-review the PR. -->
<hr>
##### Key changed/added classes (or packages if there are too many classes) in this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-922151903
> Hi, it seems that this change breaks the backward compatibility?
>
> Let's say we have 3 time series: root.sg.d.s1, root.sg.d.s2, and root.sg.d.t.s1.
>
> Before this change, the following query produces a result set that contains all of the time series.
>
> ```
> select * from root.sg.d
> ```
>
> After this change, it produces only root.sg.d.s1 and root.sg.d.s2, right?
Yes, after this change, ```select * from root.sg.d``` will be ```select ** from root.sg.d```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r716121968
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -797,35 +799,42 @@ public TSDataType getSeriesType(PartialPath path) throws MetadataException {
/**
* Get all devices under given prefixPath.
*
- * @param prefixPath a prefix of a full path. if the wildcard is not at the tail, then each
- * wildcard can only match one level, otherwise it can match to the tail.
+ * @param prefixPath a prefix of a full path or a pattern of the prefixPath.
* @return A HashSet instance which stores devices paths with given prefixPath.
*/
- public Set<PartialPath> getDevices(PartialPath prefixPath) throws MetadataException {
- return mtree.getDevices(prefixPath);
+ public Set<PartialPath> getDevicesByPrefix(PartialPath prefixPath) throws MetadataException {
+ return mtree.getDevices(prefixPath, true);
+ }
+
+ public Set<PartialPath> getDevicesForTimeseries(PartialPath timeseries) throws MetadataException {
+ return mtree.getDevicesForTimeseries(timeseries);
+ }
+
+ public Set<PartialPath> getDevices(PartialPath pathPattern) throws MetadataException {
Review comment:
Currently, all params potential to be PathPattern are named as pathPattern. The class definition and extension could be implemented in future work. The reason is described in previous conversation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713755192
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/Traverser.java
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser;
+
+import org.apache.iotdb.db.exception.metadata.IllegalPathException;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.MeasurementMNode;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.MULTI_LEVEL_PATH_WILDCARD;
+import static org.apache.iotdb.db.conf.IoTDBConstant.ONE_LEVEL_PATH_WILDCARD;
+
+// This class defines the main traversal framework and declares some methods for result process
+// extension.
+public abstract class Traverser {
+
+ protected IMNode startNode;
+ protected String[] nodes;
+
+ // if isMeasurementTraverser, measurement in template should be processed
+ protected boolean isMeasurementTraverser = false;
+
+ // default false means fullPath pattern match
+ protected boolean isPrefixMatch = false;
+
+ public Traverser(IMNode startNode, PartialPath path) throws MetadataException {
+ String[] nodes = path.getNodes();
+ if (nodes.length == 0 || !nodes[0].equals(startNode.getName())) {
+ throw new IllegalPathException(
+ path.getFullPath(), path.getFullPath() + " doesn't start with " + startNode.getName());
+ }
+ this.startNode = startNode;
+ this.nodes = nodes;
+ }
+
+ public void traverse() throws MetadataException {
+ traverse(startNode, 0, 0);
+ }
+
+ /**
+ * The recursive method for MTree traversal.
+ *
+ * @param node current node that match the targetName in given path
+ * @param idx the index of targetName in given path
+ * @param level the level of current node in MTree
+ * @throws MetadataException some result process may throw MetadataException
+ */
+ protected void traverse(IMNode node, int idx, int level) throws MetadataException {
+
+ if (processMatchedMNode(node, idx, level)) {
+ return;
+ }
+
+ if (idx >= nodes.length - 1) {
+ if (nodes[nodes.length - 1].equals(MULTI_LEVEL_PATH_WILDCARD) || isPrefixMatch) {
+ processMultiLevelWildcard(node, idx, level);
+ }
+ return;
+ }
+
+ if (node.isMeasurement()) {
+ return;
+ }
+
+ String targetName = nodes[idx + 1];
+ if (MULTI_LEVEL_PATH_WILDCARD.equals(targetName)) {
+ processMultiLevelWildcard(node, idx, level);
+ } else if (targetName.contains(ONE_LEVEL_PATH_WILDCARD)) {
+ processOneLevelWildcard(node, idx, level);
+ } else {
+ processNameMatch(node, idx, level);
+ }
+ }
+
+ // process curNode that matches the targetName during traversal
+ private boolean processMatchedMNode(IMNode node, int idx, int level) throws MetadataException {
Review comment:
javadoc added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713137937
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/collector/EntityPathCollector.java
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser.collector;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+
+import java.util.Set;
+import java.util.TreeSet;
+
+// This class implements the EntityMNode path collection function.
+public class EntityPathCollector extends CollectorTraverser<Set<PartialPath>> {
+
+ public EntityPathCollector(IMNode startNode, PartialPath path) throws MetadataException {
+ super(startNode, path);
+ this.resultSet = new TreeSet<>();
+ }
+
+ public EntityPathCollector(IMNode startNode, PartialPath path, int limit, int offset)
+ throws MetadataException {
+ super(startNode, path, limit, offset);
+ this.resultSet = new TreeSet<>();
+ }
+
+ @Override
+ protected boolean processInternalMatchedMNode(IMNode node, int idx, int level) {
+ return false;
Review comment:
need to clarify the definition of count devices root.sg.d.s, return 1 or 0
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/collector/MeasurementCollector.java
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser.collector;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.IMeasurementMNode;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+import org.apache.iotdb.tsfile.write.schema.MeasurementSchema;
+import org.apache.iotdb.tsfile.write.schema.VectorMeasurementSchema;
+
+import java.util.List;
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.MULTI_LEVEL_PATH_WILDCARD;
+
+// This class defines MeasurementMNode as target node and defines the measurement process framework.
+public abstract class MeasurementCollector<T> extends CollectorTraverser<T> {
+
+ public MeasurementCollector(IMNode startNode, PartialPath path) throws MetadataException {
+ super(startNode, path);
+ isMeasurementTraverser = true;
+ }
+
+ public MeasurementCollector(IMNode startNode, PartialPath path, int limit, int offset)
+ throws MetadataException {
+ super(startNode, path, limit, offset);
+ isMeasurementTraverser = true;
+ }
+
+ @Override
+ protected boolean processInternalMatchedMNode(IMNode node, int idx, int level)
+ throws MetadataException {
+ if (!node.isMeasurement() || idx != nodes.length - 2) {
+ return false;
+ }
+ IMeasurementSchema schema = ((IMeasurementMNode) node).getSchema();
+ if (schema instanceof VectorMeasurementSchema) {
+ List<String> measurements = schema.getSubMeasurementsList();
+ String targetNameRegex = nodes[idx + 1].replace("*", ".*");
+ for (int i = 0; i < measurements.size(); i++) {
+ if (!Pattern.matches(targetNameRegex, measurements.get(i))) {
+ continue;
+ }
+ if (hasLimit) {
+ curOffset += 1;
+ if (curOffset < offset) {
+ break;
+ }
+ }
+ collectVectorMeasurementSchema((IMeasurementMNode) node, i);
+ if (hasLimit) {
+ count += 1;
+ }
+ }
+ }
+ return true;
+ }
+
+ @Override
+ protected boolean processFullMatchedMNode(IMNode node, int idx, int level)
+ throws MetadataException {
+ if (!node.isMeasurement()) {
+ return false;
+ }
+ IMeasurementSchema schema = ((IMeasurementMNode) node).getSchema();
+ if (schema instanceof MeasurementSchema) {
+ if (hasLimit) {
+ curOffset += 1;
+ if (curOffset < offset) {
+ return true;
+ }
+ }
+ collectMeasurementSchema((IMeasurementMNode) node);
+ if (hasLimit) {
+ count += 1;
+ }
+ } else if (schema instanceof VectorMeasurementSchema) {
+ if (idx >= nodes.length - 1
+ && !nodes[nodes.length - 1].equals(MULTI_LEVEL_PATH_WILDCARD)
+ && !isPrefixMatch) {
+ return true;
+ }
+ // only when idx > nodes.length or nodes ends with ** or isPrefixMatch
+ List<String> measurements = schema.getSubMeasurementsList();
+ for (int i = 0; i < measurements.size(); i++) {
+ if (hasLimit) {
+ curOffset += 1;
+ if (curOffset < offset) {
+ return true;
+ }
+ }
+ collectVectorMeasurementSchema((IMeasurementMNode) node, i);
+ if (hasLimit) {
+ count += 1;
+ }
+ }
+ }
+ return true;
+ }
+
+ protected abstract void collectMeasurementSchema(IMeasurementMNode node) throws MetadataException;
+
+ protected abstract void collectVectorMeasurementSchema(IMeasurementMNode node, int index)
Review comment:
```suggestion
protected abstract void collectVectorMeasurementMNode(IMeasurementMNode node, int index)
```
add javadoc
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/collector/MeasurementCollector.java
##########
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser.collector;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.IMeasurementMNode;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+import org.apache.iotdb.tsfile.write.schema.MeasurementSchema;
+import org.apache.iotdb.tsfile.write.schema.VectorMeasurementSchema;
+
+import java.util.List;
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.MULTI_LEVEL_PATH_WILDCARD;
+
+// This class defines MeasurementMNode as target node and defines the measurement process framework.
+public abstract class MeasurementCollector<T> extends CollectorTraverser<T> {
+
+ public MeasurementCollector(IMNode startNode, PartialPath path) throws MetadataException {
+ super(startNode, path);
+ isMeasurementTraverser = true;
+ }
+
+ public MeasurementCollector(IMNode startNode, PartialPath path, int limit, int offset)
+ throws MetadataException {
+ super(startNode, path, limit, offset);
+ isMeasurementTraverser = true;
+ }
+
+ @Override
+ protected boolean processInternalMatchedMNode(IMNode node, int idx, int level)
+ throws MetadataException {
+ if (!node.isMeasurement() || idx != nodes.length - 2) {
+ return false;
+ }
+ IMeasurementSchema schema = ((IMeasurementMNode) node).getSchema();
+ if (schema instanceof VectorMeasurementSchema) {
+ List<String> measurements = schema.getSubMeasurementsList();
+ String targetNameRegex = nodes[idx + 1].replace("*", ".*");
+ for (int i = 0; i < measurements.size(); i++) {
+ if (!Pattern.matches(targetNameRegex, measurements.get(i))) {
+ continue;
+ }
+ if (hasLimit) {
+ curOffset += 1;
+ if (curOffset < offset) {
+ break;
+ }
+ }
+ collectVectorMeasurementSchema((IMeasurementMNode) node, i);
+ if (hasLimit) {
+ count += 1;
+ }
+ }
+ }
+ return true;
+ }
+
+ @Override
+ protected boolean processFullMatchedMNode(IMNode node, int idx, int level)
+ throws MetadataException {
+ if (!node.isMeasurement()) {
+ return false;
+ }
+ IMeasurementSchema schema = ((IMeasurementMNode) node).getSchema();
+ if (schema instanceof MeasurementSchema) {
+ if (hasLimit) {
+ curOffset += 1;
+ if (curOffset < offset) {
+ return true;
+ }
+ }
+ collectMeasurementSchema((IMeasurementMNode) node);
+ if (hasLimit) {
+ count += 1;
+ }
+ } else if (schema instanceof VectorMeasurementSchema) {
+ if (idx >= nodes.length - 1
+ && !nodes[nodes.length - 1].equals(MULTI_LEVEL_PATH_WILDCARD)
+ && !isPrefixMatch) {
+ return true;
+ }
+ // only when idx > nodes.length or nodes ends with ** or isPrefixMatch
+ List<String> measurements = schema.getSubMeasurementsList();
+ for (int i = 0; i < measurements.size(); i++) {
+ if (hasLimit) {
+ curOffset += 1;
+ if (curOffset < offset) {
+ return true;
+ }
+ }
+ collectVectorMeasurementSchema((IMeasurementMNode) node, i);
+ if (hasLimit) {
+ count += 1;
+ }
+ }
+ }
+ return true;
+ }
+
+ protected abstract void collectMeasurementSchema(IMeasurementMNode node) throws MetadataException;
Review comment:
```suggestion
protected abstract void collectMeasurementMNode(IMeasurementMNode node) throws MetadataException;
```
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/collector/StorageGroupDeterminator.java
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser.collector;
+
+import org.apache.iotdb.db.conf.IoTDBConstant;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.MULTI_LEVEL_PATH_WILDCARD;
+
+/**
+ * This class implements the storage group determination function as following description.
+ *
+ * <p>For a path, infer all storage groups it may belong to. The path can have wildcards.
+ *
+ * <p>Consider the path into two parts: (1) the sub path which can not contain a storage group name
+ * and (2) the sub path which is substring that begin after the storage group name.
+ *
+ * <p>(1) Suppose the part of the path can not contain a storage group name (e.g.,
+ * "root".contains("root.sg") == false), then: For each one level wildcard *, only one level will be
+ * inferred and the wildcard will be removed. For each multi level wildcard **, then the inference
+ * will go on until the storage groups are found and the wildcard will be kept. (2) Suppose the part
+ * of the path is a substring that begin after the storage group name. (e.g., For
+ * "root.*.sg1.a.*.b.*" and "root.x.sg1" is a storage group, then this part is "a.*.b.*"). For this
+ * part, keep what it is.
+ *
+ * <p>Assuming we have three SGs: root.group1, root.group2, root.area1.group3 Eg1: for input
+ * "root.**", returns ("root.group1", "root.group1.**"), ("root.group2", "root.group2.**")
+ * ("root.area1.group3", "root.area1.group3.**") Eg2: for input "root.*.s1", returns ("root.group1",
+ * "root.group1.s1"), ("root.group2", "root.group2.s1")
+ *
+ * <p>Eg3: for input "root.area1.**", returns ("root.area1.group3", "root.area1.group3.**")
+ *
+ * <p>ResultSet: StorageGroupName-FullPath pairs
+ */
+public class StorageGroupDeterminator extends CollectorTraverser<Map<String, String>> {
Review comment:
why using determinator...not collector
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/Traverser.java
##########
@@ -0,0 +1,210 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser;
+
+import org.apache.iotdb.db.exception.metadata.IllegalPathException;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.MeasurementMNode;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.MULTI_LEVEL_PATH_WILDCARD;
+import static org.apache.iotdb.db.conf.IoTDBConstant.ONE_LEVEL_PATH_WILDCARD;
+
+// This class defines the main traversal framework and declares some methods for result process
+// extension.
+public abstract class Traverser {
+
+ protected IMNode startNode;
+ protected String[] nodes;
+
+ // if isMeasurementTraverser, measurement in template should be processed
+ protected boolean isMeasurementTraverser = false;
+
+ // default false means fullPath pattern match
+ protected boolean isPrefixMatch = false;
+
+ public Traverser(IMNode startNode, PartialPath path) throws MetadataException {
+ String[] nodes = path.getNodes();
+ if (nodes.length == 0 || !nodes[0].equals(startNode.getName())) {
+ throw new IllegalPathException(
+ path.getFullPath(), path.getFullPath() + " doesn't start with " + startNode.getName());
+ }
+ this.startNode = startNode;
+ this.nodes = nodes;
+ }
+
+ public void traverse() throws MetadataException {
+ traverse(startNode, 0, 0);
+ }
+
+ /**
+ * The recursive method for MTree traversal.
+ *
+ * @param node current node that match the targetName in given path
+ * @param idx the index of targetName in given path
+ * @param level the level of current node in MTree
+ * @throws MetadataException some result process may throw MetadataException
+ */
+ protected void traverse(IMNode node, int idx, int level) throws MetadataException {
+
+ if (processMatchedMNode(node, idx, level)) {
+ return;
+ }
+
+ if (idx >= nodes.length - 1) {
+ if (nodes[nodes.length - 1].equals(MULTI_LEVEL_PATH_WILDCARD) || isPrefixMatch) {
+ processMultiLevelWildcard(node, idx, level);
+ }
+ return;
+ }
+
+ if (node.isMeasurement()) {
+ return;
+ }
+
+ String targetName = nodes[idx + 1];
+ if (MULTI_LEVEL_PATH_WILDCARD.equals(targetName)) {
+ processMultiLevelWildcard(node, idx, level);
+ } else if (targetName.contains(ONE_LEVEL_PATH_WILDCARD)) {
+ processOneLevelWildcard(node, idx, level);
+ } else {
+ processNameMatch(node, idx, level);
+ }
+ }
+
+ // process curNode that matches the targetName during traversal
+ private boolean processMatchedMNode(IMNode node, int idx, int level) throws MetadataException {
Review comment:
add javadoc for return value or remove this method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r709807393
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -207,7 +207,7 @@ IoTDB supports the 'order by time' statement since 0.11, it's used to display re
For example, the SQL statement is:
```sql
-select * from root.ln where time > 1 order by time desc limit 10;
+select * from root.ln.* where time > 1 order by time desc limit 10;
Review comment:
The exampled SQL statements in IoTDB Doc seem to explain operations in a standard four layer scenario, root-sg-entity-measurement. If so, * or ** in this sql may bring same resultset.
Should we add a scenario declaration in our DDL、DML and SQL docs? I think this may make it more understandable and reduce ambiguity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42880784)
Coverage decreased (-0.04%) to 67.353% when pulling **360b54499feb764ff12d17afa563896c4fe4235e on zyk990424:wildcard_extension** into **511cb00e948a3dfa9d435daf8ffa4ba9a5d9317f on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42983401)
Coverage decreased (-0.1%) to 67.427% when pulling **000b27afb52e69e5f16d05b5c4b4cddd087e8593 on zyk990424:wildcard_extension** into **4e778b56900a177b35aa4c8fda8a175eb417313a on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43070362)
Coverage increased (+0.01%) to 67.521% when pulling **cb217bdbf6883afc65197d41bc740d805411edc5 on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43040210)
Coverage increased (+0.04%) to 67.548% when pulling **0b80f2414ec599dd3b29c52e61bcb3cdea862645 on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713754791
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/Traverser.java
##########
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser;
+
+import org.apache.iotdb.db.exception.metadata.IllegalPathException;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.MManager.StorageGroupFilter;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.MeasurementMNode;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_MULTI_LEVEL_WILDCARD;
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_ONE_LEVEL_WILDCARD;
+
+// This class defines the main traversal framework and declares some methods for result process
+// extension.
+public abstract class Traverser {
+
+ protected IMNode startNode;
+ protected String[] nodes;
+
+ // if isMeasurementTraverser, measurement in template should be processed
+ protected boolean isMeasurementTraverser = false;
+
+ // default false means fullPath pattern match
+ protected boolean isPrefixMatch = false;
+
+ // level query option
+ protected boolean isLevelTraverser = false;
+ protected int targetLevel;
+
+ // traverse for specific storage group
+ protected StorageGroupFilter storageGroupFilter = null;
+
+ public Traverser(IMNode startNode, PartialPath path) throws MetadataException {
+ String[] nodes = path.getNodes();
+ if (nodes.length == 0 || !nodes[0].equals(startNode.getName())) {
+ throw new IllegalPathException(
+ path.getFullPath(), path.getFullPath() + " doesn't start with " + startNode.getName());
+ }
+ this.startNode = startNode;
+ this.nodes = nodes;
+ }
+
+ public void traverse() throws MetadataException {
Review comment:
This class has nothing to do with task results. This class only defines the universal traversal framework. I've added some javadoc about extension to this method and class.
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/Traverser.java
##########
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser;
+
+import org.apache.iotdb.db.exception.metadata.IllegalPathException;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.MManager.StorageGroupFilter;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.MeasurementMNode;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_MULTI_LEVEL_WILDCARD;
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_ONE_LEVEL_WILDCARD;
+
+// This class defines the main traversal framework and declares some methods for result process
+// extension.
+public abstract class Traverser {
+
+ protected IMNode startNode;
+ protected String[] nodes;
+
+ // if isMeasurementTraverser, measurement in template should be processed
+ protected boolean isMeasurementTraverser = false;
+
+ // default false means fullPath pattern match
+ protected boolean isPrefixMatch = false;
+
+ // level query option
+ protected boolean isLevelTraverser = false;
+ protected int targetLevel;
+
+ // traverse for specific storage group
+ protected StorageGroupFilter storageGroupFilter = null;
+
+ public Traverser(IMNode startNode, PartialPath path) throws MetadataException {
+ String[] nodes = path.getNodes();
+ if (nodes.length == 0 || !nodes[0].equals(startNode.getName())) {
+ throw new IllegalPathException(
+ path.getFullPath(), path.getFullPath() + " doesn't start with " + startNode.getName());
+ }
+ this.startNode = startNode;
+ this.nodes = nodes;
+ }
+
+ public void traverse() throws MetadataException {
+ traverse(startNode, 0, false, 0);
+ }
+
+ /**
+ * The recursive method for MTree traversal.
+ *
+ * @param node current node that match the targetName in given path
+ * @param idx the index of targetName in given path
+ * @param multiLevelWildcard whether the current targetName is **
+ * @param level the level of current node in MTree
+ * @throws MetadataException some result process may throw MetadataException
+ */
+ protected void traverse(IMNode node, int idx, boolean multiLevelWildcard, int level)
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r709824829
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -207,7 +207,7 @@ IoTDB supports the 'order by time' statement since 0.11, it's used to display re
For example, the SQL statement is:
```sql
-select * from root.ln where time > 1 order by time desc limit 10;
+select * from root.ln.* where time > 1 order by time desc limit 10;
Review comment:
Fixed.
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -214,7 +214,7 @@ It costs 0.014s
* 根据时间降序返回
IoTDB 在 0.11 版本开始支持 'order by time' 语句，用于对结果按照时间进行降序展示。例如，SQL 语句为：
```sql
-select * from root.ln where time > 1 order by time desc limit 10;
+select * from root.ln.* where time > 1 order by time desc limit 10;
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r710073848
##########
File path: cross-tests/src/test/java/org/apache/iotdb/cross/tests/tools/importCsv/ImportCsvTestIT.java
##########
@@ -155,7 +155,7 @@ public void testWithoutCreateSchema() throws IOException, ClassNotFoundException
DriverManager.getConnection(
Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
Statement statement = connection.createStatement()) {
- if (statement.execute("select * from root")) {
+ if (statement.execute("select * from root.**")) {
Review comment:
change all `select * from root.**` to `select ** from root`
##########
File path: cluster/src/main/java/org/apache/iotdb/cluster/metadata/CMManager.java
##########
@@ -1204,14 +1204,14 @@ public void setCoordinator(Coordinator coordinator) {
/** Similar to method getAllTimeseriesPath(), but return Path with alias alias. */
@Override
public Pair<List<PartialPath>, Integer> getAllTimeseriesPathWithAlias(
- PartialPath prefixPath, int limit, int offset) throws MetadataException {
+ PartialPath path, int limit, int offset) throws MetadataException {
// get all storage groups this path may belong to
// the key is the storage group name and the value is the path to be queried with storage group
// added, e.g:
// "root.*" will be translated into:
// "root.group1" -> "root.group1.*", "root.group2" -> "root.group2.*" ...
Review comment:
I cannot understand this...
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/PartialPath.java
##########
@@ -136,24 +140,49 @@ public PartialPath alterPrefixPath(PartialPath prefixPath) {
}
/**
- * Test if this PartialPath matches a full path. rPath is supposed to be a full timeseries path
- * without wildcards. e.g. "root.sg.device.*" matches path "root.sg.device.s1" whereas it does not
- * match "root.sg.device" and "root.sg.vehicle.s1"
+ * Test if this PartialPath matches a full path. This partialPath acts as a full path pattern.
+ * rPath is supposed to be a full timeseries path without wildcards. e.g. "root.sg.device.*"
+ * matches path "root.sg.device.s1" whereas it does not match "root.sg.device" and
+ * "root.sg.vehicle.s1"
*
* @param rPath a plain full path of a timeseries
* @return true if a successful match, otherwise return false
*/
public boolean matchFullPath(PartialPath rPath) {
- String[] rNodes = rPath.getNodes();
- if (rNodes.length < nodes.length) {
+ return matchFullPath(rPath.getNodes(), 0, 0, false);
+ }
+
+ private boolean matchFullPath(
+ String[] pathNodes, int pathIndex, int patternIndex, boolean multiLevelWild) {
+ if (pathIndex == pathNodes.length && patternIndex == nodes.length) {
+ return true;
+ } else if (patternIndex == nodes.length && multiLevelWild) {
+ return matchFullPath(pathNodes, pathIndex + 1, patternIndex, true);
+ } else if (pathIndex >= pathNodes.length || patternIndex >= nodes.length) {
return false;
}
- for (int i = 0; i < nodes.length; i++) {
- if (!nodes[i].equals(IoTDBConstant.PATH_WILDCARD) && !nodes[i].equals(rNodes[i])) {
- return false;
+
+ String pathNode = pathNodes[pathIndex];
+ String patternNode = nodes[patternIndex];
+ boolean isMatch = false;
+ if (patternNode.equals(PATH_MULTI_LEVEL_WILDCARD)) {
+ isMatch = matchFullPath(pathNodes, pathIndex + 1, patternIndex + 1, true);
+ } else {
+ if (patternNode.contains(PATH_ONE_LEVEL_WILDCARD)) {
Review comment:
add a test case:
root.sg.device*
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -797,35 +799,42 @@ public TSDataType getSeriesType(PartialPath path) throws MetadataException {
/**
* Get all devices under given prefixPath.
*
- * @param prefixPath a prefix of a full path. if the wildcard is not at the tail, then each
- * wildcard can only match one level, otherwise it can match to the tail.
+ * @param prefixPath a prefix of a full path or a pattern of the prefixPath.
* @return A HashSet instance which stores devices paths with given prefixPath.
*/
- public Set<PartialPath> getDevices(PartialPath prefixPath) throws MetadataException {
- return mtree.getDevices(prefixPath);
+ public Set<PartialPath> getDevicesByPrefix(PartialPath prefixPath) throws MetadataException {
+ return mtree.getDevices(prefixPath, true);
+ }
+
+ public Set<PartialPath> getDevicesForTimeseries(PartialPath timeseries) throws MetadataException {
+ return mtree.getDevicesForTimeseries(timeseries);
+ }
+
+ public Set<PartialPath> getDevices(PartialPath pathPattern) throws MetadataException {
Review comment:
could we separate pathPattern from PartialPath by define two class?
##########
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
##########
@@ -115,7 +115,8 @@ private IoTDBConstant() {}
public static final String COLUMN_TRIGGER_STATUS_STARTED = "started";
public static final String COLUMN_TRIGGER_STATUS_STOPPED = "stopped";
- public static final String PATH_WILDCARD = "*";
+ public static final String PATH_ONE_LEVEL_WILDCARD = "*";
+ public static final String PATH_MULTI_LEVEL_WILDCARD = "**";
Review comment:
```suggestion
public static final String MULTI_LEVEL_PATH_WILDCARD = "**";
```
##########
File path: cluster/src/main/java/org/apache/iotdb/cluster/metadata/CMManager.java
##########
@@ -1204,14 +1204,14 @@ public void setCoordinator(Coordinator coordinator) {
/** Similar to method getAllTimeseriesPath(), but return Path with alias alias. */
@Override
public Pair<List<PartialPath>, Integer> getAllTimeseriesPathWithAlias(
- PartialPath prefixPath, int limit, int offset) throws MetadataException {
+ PartialPath path, int limit, int offset) throws MetadataException {
// get all storage groups this path may belong to
// the key is the storage group name and the value is the path to be queried with storage group
// added, e.g:
// "root.*" will be translated into:
// "root.group1" -> "root.group1.*", "root.group2" -> "root.group2.*" ...
- Map<String, String> sgPathMap = determineStorageGroup(prefixPath);
+ Map<String, String> sgPathMap = determineStorageGroup(path);
Review comment:
better to change determinStorageGroup to getStorageGroups or so
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/MManager.java
##########
@@ -797,35 +799,42 @@ public TSDataType getSeriesType(PartialPath path) throws MetadataException {
/**
* Get all devices under given prefixPath.
*
- * @param prefixPath a prefix of a full path. if the wildcard is not at the tail, then each
- * wildcard can only match one level, otherwise it can match to the tail.
+ * @param prefixPath a prefix of a full path or a pattern of the prefixPath.
* @return A HashSet instance which stores devices paths with given prefixPath.
*/
- public Set<PartialPath> getDevices(PartialPath prefixPath) throws MetadataException {
- return mtree.getDevices(prefixPath);
+ public Set<PartialPath> getDevicesByPrefix(PartialPath prefixPath) throws MetadataException {
+ return mtree.getDevices(prefixPath, true);
+ }
+
+ public Set<PartialPath> getDevicesForTimeseries(PartialPath timeseries) throws MetadataException {
Review comment:
add javadoc, can not get the meaning of this method, getDevicesByTimeseries?
##########
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
##########
@@ -115,7 +115,8 @@ private IoTDBConstant() {}
public static final String COLUMN_TRIGGER_STATUS_STARTED = "started";
public static final String COLUMN_TRIGGER_STATUS_STOPPED = "stopped";
- public static final String PATH_WILDCARD = "*";
+ public static final String PATH_ONE_LEVEL_WILDCARD = "*";
Review comment:
```suggestion
public static final String ONE_LEVEL_PATH_WILDCARD = "*";
```
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/Traverser.java
##########
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser;
+
+import org.apache.iotdb.db.exception.metadata.IllegalPathException;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.MManager.StorageGroupFilter;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.MeasurementMNode;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_MULTI_LEVEL_WILDCARD;
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_ONE_LEVEL_WILDCARD;
+
+// This class defines the main traversal framework and declares some methods for result process
+// extension.
+public abstract class Traverser {
+
+ protected IMNode startNode;
+ protected String[] nodes;
+
+ // if isMeasurementTraverser, measurement in template should be processed
+ protected boolean isMeasurementTraverser = false;
+
+ // default false means fullPath pattern match
+ protected boolean isPrefixMatch = false;
+
+ // level query option
+ protected boolean isLevelTraverser = false;
+ protected int targetLevel;
+
+ // traverse for specific storage group
+ protected StorageGroupFilter storageGroupFilter = null;
+
+ public Traverser(IMNode startNode, PartialPath path) throws MetadataException {
+ String[] nodes = path.getNodes();
+ if (nodes.length == 0 || !nodes[0].equals(startNode.getName())) {
+ throw new IllegalPathException(
+ path.getFullPath(), path.getFullPath() + " doesn't start with " + startNode.getName());
+ }
+ this.startNode = startNode;
+ this.nodes = nodes;
+ }
+
+ public void traverse() throws MetadataException {
Review comment:
add javadoc, How to get the result.
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/Traverser.java
##########
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser;
+
+import org.apache.iotdb.db.exception.metadata.IllegalPathException;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.MManager.StorageGroupFilter;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+import org.apache.iotdb.db.metadata.mnode.MeasurementMNode;
+import org.apache.iotdb.db.metadata.template.Template;
+import org.apache.iotdb.tsfile.write.schema.IMeasurementSchema;
+
+import java.util.regex.Pattern;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_MULTI_LEVEL_WILDCARD;
+import static org.apache.iotdb.db.conf.IoTDBConstant.PATH_ONE_LEVEL_WILDCARD;
+
+// This class defines the main traversal framework and declares some methods for result process
+// extension.
+public abstract class Traverser {
+
+ protected IMNode startNode;
+ protected String[] nodes;
+
+ // if isMeasurementTraverser, measurement in template should be processed
+ protected boolean isMeasurementTraverser = false;
+
+ // default false means fullPath pattern match
+ protected boolean isPrefixMatch = false;
+
+ // level query option
+ protected boolean isLevelTraverser = false;
+ protected int targetLevel;
+
+ // traverse for specific storage group
+ protected StorageGroupFilter storageGroupFilter = null;
+
+ public Traverser(IMNode startNode, PartialPath path) throws MetadataException {
+ String[] nodes = path.getNodes();
+ if (nodes.length == 0 || !nodes[0].equals(startNode.getName())) {
+ throw new IllegalPathException(
+ path.getFullPath(), path.getFullPath() + " doesn't start with " + startNode.getName());
+ }
+ this.startNode = startNode;
+ this.nodes = nodes;
+ }
+
+ public void traverse() throws MetadataException {
+ traverse(startNode, 0, false, 0);
+ }
+
+ /**
+ * The recursive method for MTree traversal.
+ *
+ * @param node current node that match the targetName in given path
+ * @param idx the index of targetName in given path
+ * @param multiLevelWildcard whether the current targetName is **
+ * @param level the level of current node in MTree
+ * @throws MetadataException some result process may throw MetadataException
+ */
+ protected void traverse(IMNode node, int idx, boolean multiLevelWildcard, int level)
Review comment:
add javadoc: if the node matches nodes[idx], then do some operation and traverse the children with nodes[idx+1]
// intermal match: root.sg internal match root.sg.**(pattern)
boolean processInternalMatchedMNode(IMNode node)
// full match: root.sg.d full match root.sg.**(pattern)
boolean processFullMatchedMNode(IMNode node)
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/PartialPath.java
##########
@@ -32,6 +32,10 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
+import java.util.regex.Pattern;
+
Review comment:
maybe changing PartialPath to PathPattern is better
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42914436)
Coverage decreased (-0.01%) to 67.377% when pulling **54bf301f82a5c5402ece0b1dce88ba5de68d905f on zyk990424:wildcard_extension** into **511cb00e948a3dfa9d435daf8ffa4ba9a5d9317f on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] cornmonster commented on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
cornmonster commented on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-921404440
Hi, it seems that this change breaks the backward compatibility?
Let's say we have 3 time series: root.sg.d.s1, root.sg.d.s2, and root.sg.d.t.s1.
Before this change, the following query produces a result set that contains all of the time series.
```
select * from root.sg.d
```
After this change, it produces only root.sg.d.s1 and root.sg.d.s2, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls commented on pull request #3918: Wildcard extension
Posted by GitBox <gi...@apache.org>.
coveralls commented on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42687196)
Coverage decreased (-0.001%) to 67.375% when pulling **a42635941529b98f3d2bb80bcad7d05f1fad3073 on zyk990424:wildcard_extension** into **1ad4b3dcdc3a99a71254b18561e64fdbfa6b79db on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43069140)
Coverage increased (+0.07%) to 67.575% when pulling **49d19b9aa78c1900981dd47434867dbbfe660cd5 on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r716121814
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/PartialPath.java
##########
@@ -32,6 +32,10 @@
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
+import java.util.regex.Pattern;
+
Review comment:
It's a good idea. I think there should be a fence that separate the PathPattern usage and PartialPath usage. Maybe the first interaction between QueryPlanExecutor and MManager could act as this fence. This interaction will parse all PathPattern into PartialPath set. However, some implementation of functions, like delete, will keep using PathPattern for several times during execution. Maybe we could fix this in next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42855217)
Coverage increased (+0.1%) to 67.508% when pulling **753d86f40cc3d5a0605da37946ac8494a9bc0170 on zyk990424:wildcard_extension** into **511cb00e948a3dfa9d435daf8ffa4ba9a5d9317f on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713757784
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/collector/StorageGroupDeterminator.java
##########
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser.collector;
+
+import org.apache.iotdb.db.conf.IoTDBConstant;
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.apache.iotdb.db.conf.IoTDBConstant.MULTI_LEVEL_PATH_WILDCARD;
+
+/**
+ * This class implements the storage group determination function as following description.
+ *
+ * <p>For a path, infer all storage groups it may belong to. The path can have wildcards.
+ *
+ * <p>Consider the path into two parts: (1) the sub path which can not contain a storage group name
+ * and (2) the sub path which is substring that begin after the storage group name.
+ *
+ * <p>(1) Suppose the part of the path can not contain a storage group name (e.g.,
+ * "root".contains("root.sg") == false), then: For each one level wildcard *, only one level will be
+ * inferred and the wildcard will be removed. For each multi level wildcard **, then the inference
+ * will go on until the storage groups are found and the wildcard will be kept. (2) Suppose the part
+ * of the path is a substring that begin after the storage group name. (e.g., For
+ * "root.*.sg1.a.*.b.*" and "root.x.sg1" is a storage group, then this part is "a.*.b.*"). For this
+ * part, keep what it is.
+ *
+ * <p>Assuming we have three SGs: root.group1, root.group2, root.area1.group3 Eg1: for input
+ * "root.**", returns ("root.group1", "root.group1.**"), ("root.group2", "root.group2.**")
+ * ("root.area1.group3", "root.area1.group3.**") Eg2: for input "root.*.s1", returns ("root.group1",
+ * "root.group1.s1"), ("root.group2", "root.group2.s1")
+ *
+ * <p>Eg3: for input "root.area1.**", returns ("root.area1.group3", "root.area1.group3.**")
+ *
+ * <p>ResultSet: StorageGroupName-FullPath pairs
+ */
+public class StorageGroupDeterminator extends CollectorTraverser<Map<String, String>> {
Review comment:
Renamed to StorageGroupResolver, since the task of this class is to resolve given path into SG-FullPath pairs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713756696
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/mtree/traverser/collector/EntityPathCollector.java
##########
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.iotdb.db.metadata.mtree.traverser.collector;
+
+import org.apache.iotdb.db.exception.metadata.MetadataException;
+import org.apache.iotdb.db.metadata.PartialPath;
+import org.apache.iotdb.db.metadata.mnode.IMNode;
+
+import java.util.Set;
+import java.util.TreeSet;
+
+// This class implements the EntityMNode path collection function.
+public class EntityPathCollector extends CollectorTraverser<Set<PartialPath>> {
+
+ public EntityPathCollector(IMNode startNode, PartialPath path) throws MetadataException {
+ super(startNode, path);
+ this.resultSet = new TreeSet<>();
+ }
+
+ public EntityPathCollector(IMNode startNode, PartialPath path, int limit, int offset)
+ throws MetadataException {
+ super(startNode, path, limit, offset);
+ this.resultSet = new TreeSet<>();
+ }
+
+ @Override
+ protected boolean processInternalMatchedMNode(IMNode node, int idx, int level) {
+ return false;
Review comment:
Return 1 or 0 means two different tasks. I process them separately in EntityPathCollector and TSEntityPathCollector. I've added comments on the differences between two classes in EntityPathCollector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
HTHou commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r709693388
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -207,7 +207,7 @@ IoTDB supports the 'order by time' statement since 0.11, it's used to display re
For example, the SQL statement is:
```sql
-select * from root.ln where time > 1 order by time desc limit 10;
+select * from root.ln.* where time > 1 order by time desc limit 10;
Review comment:
** or *?
##########
File path: docs/zh/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -214,7 +214,7 @@ It costs 0.014s
* 根据时间降序返回
IoTDB 在 0.11 版本开始支持 'order by time' 语句，用于对结果按照时间进行降序展示。例如，SQL 语句为：
```sql
-select * from root.ln where time > 1 order by time desc limit 10;
+select * from root.ln.* where time > 1 order by time desc limit 10;
Review comment:
** or * ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43069367)
Coverage increased (+0.04%) to 67.546% when pulling **6ecb182ba9524d1504c49cdcd66b46ecb35eb71f on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42992350)
Coverage decreased (-0.01%) to 67.419% when pulling **e9fe80e1d4e01274910448646393cd6628ee540a on zyk990424:wildcard_extension** into **c75a71f76c10a3f79b905df3daeef5429704a7f7 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713752561
##########
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
##########
@@ -115,7 +115,8 @@ private IoTDBConstant() {}
public static final String COLUMN_TRIGGER_STATUS_STARTED = "started";
public static final String COLUMN_TRIGGER_STATUS_STOPPED = "stopped";
- public static final String PATH_WILDCARD = "*";
+ public static final String PATH_ONE_LEVEL_WILDCARD = "*";
Review comment:
Fixed.
##########
File path: server/src/main/java/org/apache/iotdb/db/conf/IoTDBConstant.java
##########
@@ -115,7 +115,8 @@ private IoTDBConstant() {}
public static final String COLUMN_TRIGGER_STATUS_STARTED = "started";
public static final String COLUMN_TRIGGER_STATUS_STOPPED = "stopped";
- public static final String PATH_WILDCARD = "*";
+ public static final String PATH_ONE_LEVEL_WILDCARD = "*";
+ public static final String PATH_MULTI_LEVEL_WILDCARD = "**";
Review comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713752382
##########
File path: cluster/src/main/java/org/apache/iotdb/cluster/metadata/CMManager.java
##########
@@ -1204,14 +1204,14 @@ public void setCoordinator(Coordinator coordinator) {
/** Similar to method getAllTimeseriesPath(), but return Path with alias alias. */
@Override
public Pair<List<PartialPath>, Integer> getAllTimeseriesPathWithAlias(
- PartialPath prefixPath, int limit, int offset) throws MetadataException {
+ PartialPath path, int limit, int offset) throws MetadataException {
// get all storage groups this path may belong to
// the key is the storage group name and the value is the path to be queried with storage group
// added, e.g:
// "root.*" will be translated into:
// "root.group1" -> "root.group1.*", "root.group2" -> "root.group2.*" ...
Review comment:
I've updated the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43070840)
Coverage increased (+0.004%) to 67.545% when pulling **f0e42974ee1891e435310757ca01176225433190 on zyk990424:wildcard_extension** into **a9f582e723ebcbed559cc6602442800c4fbbc2e3 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43041599)
Coverage increased (+0.03%) to 67.54% when pulling **60879e7ebc7345ca076afdc88a42634be2b1bef1 on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: Wildcard extension
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42691790)
Coverage increased (+0.1%) to 67.522% when pulling **6f1468420954e20776da8100c357a70a21aed400 on zyk990424:wildcard_extension** into **1ad4b3dcdc3a99a71254b18561e64fdbfa6b79db on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42884818)
Coverage increased (+0.004%) to 67.394% when pulling **ecf6ed190470b82d10cd25ec5003d8e858f0bde2 on zyk990424:wildcard_extension** into **511cb00e948a3dfa9d435daf8ffa4ba9a5d9317f on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43070745)
Coverage decreased (-0.01%) to 67.528% when pulling **f0e42974ee1891e435310757ca01176225433190 on zyk990424:wildcard_extension** into **a9f582e723ebcbed559cc6602442800c4fbbc2e3 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713753053
##########
File path: server/src/main/java/org/apache/iotdb/db/metadata/PartialPath.java
##########
@@ -136,24 +140,49 @@ public PartialPath alterPrefixPath(PartialPath prefixPath) {
}
/**
- * Test if this PartialPath matches a full path. rPath is supposed to be a full timeseries path
- * without wildcards. e.g. "root.sg.device.*" matches path "root.sg.device.s1" whereas it does not
- * match "root.sg.device" and "root.sg.vehicle.s1"
+ * Test if this PartialPath matches a full path. This partialPath acts as a full path pattern.
+ * rPath is supposed to be a full timeseries path without wildcards. e.g. "root.sg.device.*"
+ * matches path "root.sg.device.s1" whereas it does not match "root.sg.device" and
+ * "root.sg.vehicle.s1"
*
* @param rPath a plain full path of a timeseries
* @return true if a successful match, otherwise return false
*/
public boolean matchFullPath(PartialPath rPath) {
- String[] rNodes = rPath.getNodes();
- if (rNodes.length < nodes.length) {
+ return matchFullPath(rPath.getNodes(), 0, 0, false);
+ }
+
+ private boolean matchFullPath(
+ String[] pathNodes, int pathIndex, int patternIndex, boolean multiLevelWild) {
+ if (pathIndex == pathNodes.length && patternIndex == nodes.length) {
+ return true;
+ } else if (patternIndex == nodes.length && multiLevelWild) {
+ return matchFullPath(pathNodes, pathIndex + 1, patternIndex, true);
+ } else if (pathIndex >= pathNodes.length || patternIndex >= nodes.length) {
return false;
}
- for (int i = 0; i < nodes.length; i++) {
- if (!nodes[i].equals(IoTDBConstant.PATH_WILDCARD) && !nodes[i].equals(rNodes[i])) {
- return false;
+
+ String pathNode = pathNodes[pathIndex];
+ String patternNode = nodes[patternIndex];
+ boolean isMatch = false;
+ if (patternNode.equals(PATH_MULTI_LEVEL_WILDCARD)) {
+ isMatch = matchFullPath(pathNodes, pathIndex + 1, patternIndex + 1, true);
+ } else {
+ if (patternNode.contains(PATH_ONE_LEVEL_WILDCARD)) {
Review comment:
I've add a test case in PartialPathTest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r713751903
##########
File path: cluster/src/main/java/org/apache/iotdb/cluster/metadata/CMManager.java
##########
@@ -1204,14 +1204,14 @@ public void setCoordinator(Coordinator coordinator) {
/** Similar to method getAllTimeseriesPath(), but return Path with alias alias. */
@Override
public Pair<List<PartialPath>, Integer> getAllTimeseriesPathWithAlias(
- PartialPath prefixPath, int limit, int offset) throws MetadataException {
+ PartialPath path, int limit, int offset) throws MetadataException {
// get all storage groups this path may belong to
// the key is the storage group name and the value is the path to be queried with storage group
// added, e.g:
// "root.*" will be translated into:
// "root.group1" -> "root.group1.*", "root.group2" -> "root.group2.*" ...
- Map<String, String> sgPathMap = determineStorageGroup(prefixPath);
+ Map<String, String> sgPathMap = determineStorageGroup(path);
Review comment:
According to the effect of this method, resolve given path by storage group into SG-FullPath pairs, I've renamed this method into resolvePathByStorageGroup
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42855424)
Coverage decreased (-0.02%) to 67.373% when pulling **753d86f40cc3d5a0605da37946ac8494a9bc0170 on zyk990424:wildcard_extension** into **511cb00e948a3dfa9d435daf8ffa4ba9a5d9317f on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r709807393
##########
File path: docs/UserGuide/IoTDB-SQL-Language/DML-Data-Manipulation-Language.md
##########
@@ -207,7 +207,7 @@ IoTDB supports the 'order by time' statement since 0.11, it's used to display re
For example, the SQL statement is:
```sql
-select * from root.ln where time > 1 order by time desc limit 10;
+select * from root.ln.* where time > 1 order by time desc limit 10;
Review comment:
The exampled SQL statements in IoTDB Doc seem to explain operations in a standard four layer scenario, root-sg-entity-measurement. If so, * or ** in this sql may bring same resultset.
Should we add a scenario declaration in our DDL、DML and SQL docs? I think this may make it more understandable and reduce ambiguity.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/42884584)
Coverage increased (+0.09%) to 67.483% when pulling **ecf6ed190470b82d10cd25ec5003d8e858f0bde2 on zyk990424:wildcard_extension** into **511cb00e948a3dfa9d435daf8ffa4ba9a5d9317f on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43069506)
Coverage increased (+0.04%) to 67.546% when pulling **c56c7c8f6da975a9332feba829ff3dd098b87682 on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] zyk990424 commented on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
zyk990424 commented on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-921415400
> Hi, it seems that this change breaks the backward compatibility?
>
> Let's say we have 3 time series: root.sg.d.s1, root.sg.d.s2, and root.sg.d.t.s1.
>
> Before this change, the following query produces a result set that contains all of the time series.
>
> ```
> select * from root.sg.d
> ```
>
> After this change, it produces only root.sg.d.s1 and root.sg.d.s2, right?
Yes. It's a big sematic change.
In this new feature, you may use the following sql to get all the timeseries.
```
select ** from root.sg.d
select * from root.sg.d, root.sg.d.**
```
The select clause and from clause will be concat to get the path pattern of timeseries.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin merged pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] qiaojialin commented on a change in pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
qiaojialin commented on a change in pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#discussion_r710073848
##########
File path: cross-tests/src/test/java/org/apache/iotdb/cross/tests/tools/importCsv/ImportCsvTestIT.java
##########
@@ -155,7 +155,7 @@ public void testWithoutCreateSchema() throws IOException, ClassNotFoundException
DriverManager.getConnection(
Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root");
Statement statement = connection.createStatement()) {
- if (statement.execute("select * from root")) {
+ if (statement.execute("select * from root.**")) {
Review comment:
change all `select * from root.**` to `select ** from root`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] coveralls edited a comment on pull request #3918: [IOTDB-1026] Support wildcard ** in Path And Replace PrefixPath usage with PathPattern in IOTDB-SQL
Posted by GitBox <gi...@apache.org>.
coveralls edited a comment on pull request #3918:
URL: https://github.com/apache/iotdb/pull/3918#issuecomment-913958589
[](https://coveralls.io/builds/43018147)
Coverage increased (+0.02%) to 67.526% when pulling **0b22af6dca20a353c61d0a124fe4afc1a8e384f4 on zyk990424:wildcard_extension** into **ffadb30906d04632cf495e596a3718b90efb4d78 on apache:master**.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@iotdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org