You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2020/08/24 11:23:52 UTC

[GitHub] [incubator-iotdb] JackieTien97 opened a new pull request #1651: [IOTDB-849] Optimize value filter query

JackieTien97 opened a new pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651


   ![image](https://user-images.githubusercontent.com/16079446/91039486-397c5880-e63f-11ea-8665-b36bc16209a7.png)
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-iotdb] sonarcloud[bot] removed a comment on pull request #1651: [IOTDB-849] Optimize value filter query

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] removed a comment on pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651#issuecomment-679084155


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/i
 ssues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo.png' alt='No Coverage information' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651) No Coverage information  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-iotdb] Alima777 commented on a change in pull request #1651: [IOTDB-849] Optimize value filter query

Posted by GitBox <gi...@apache.org>.
Alima777 commented on a change in pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651#discussion_r477250472



##########
File path: server/src/main/java/org/apache/iotdb/db/query/timegenerator/ServerTimeGenerator.java
##########
@@ -65,19 +67,42 @@ public ServerTimeGenerator(IExpression expression, QueryContext context, RawData
   @Override
   protected IBatchReader generateNewBatchReader(SingleSeriesExpression expression)
       throws IOException {
-    Filter filter = expression.getFilter();
+    Filter valueFilter = expression.getFilter();
     Path path = expression.getSeriesPath();
     TSDataType dataType;
     QueryDataSource queryDataSource;
     try {
       dataType = IoTDB.metaManager.getSeriesType(path.getFullPath());
-      queryDataSource = QueryResourceManager.getInstance().getQueryDataSource(path, context, filter);
-      // update filter by TTL
-      filter = queryDataSource.updateFilterUsingTTL(filter);
+      queryDataSource = QueryResourceManager.getInstance().getQueryDataSource(path, context, valueFilter);
+      // update valueFilter by TTL
+      valueFilter = queryDataSource.updateFilterUsingTTL(valueFilter);
     } catch (Exception e) {
       throw new IOException(e);
     }
 
-    return new SeriesRawDataBatchReader(path, queryPlan.getAllMeasurementsInDevice(path.getDevice()), dataType, context, queryDataSource, null, filter, null);
+    // get the TimeFilter part in SingleSeriesExpression
+    Filter timeFilter = getTimeFilter(valueFilter);
+
+    return new SeriesRawDataBatchReader(path, queryPlan.getAllMeasurementsInDevice(path.getDevice()), dataType, context, queryDataSource, timeFilter, valueFilter, null);
+  }
+
+  /**
+   * extract time filter from a value filter
+   */
+  private Filter getTimeFilter(Filter filter) {
+    if (filter instanceof UnaryFilter && ((UnaryFilter) filter).getFilterType() == FilterType.TIME_FILTER) {
+      return filter;
+    }
+    if (filter instanceof AndFilter) {
+      Filter leftTimeFilter = getTimeFilter(((AndFilter) filter).getLeft());
+      Filter rightTimeFilter = getTimeFilter(((AndFilter) filter).getRight());
+      if (leftTimeFilter != null && rightTimeFilter != null) {
+        return filter;
+      } else if (leftTimeFilter != null) {
+        return leftTimeFilter;
+      } else

Review comment:
       Missing { } here?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-iotdb] JackieTien97 commented on a change in pull request #1651: [IOTDB-849] Optimize value filter query

Posted by GitBox <gi...@apache.org>.
JackieTien97 commented on a change in pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651#discussion_r477269148



##########
File path: server/src/main/java/org/apache/iotdb/db/query/timegenerator/ServerTimeGenerator.java
##########
@@ -65,19 +67,42 @@ public ServerTimeGenerator(IExpression expression, QueryContext context, RawData
   @Override
   protected IBatchReader generateNewBatchReader(SingleSeriesExpression expression)
       throws IOException {
-    Filter filter = expression.getFilter();
+    Filter valueFilter = expression.getFilter();
     Path path = expression.getSeriesPath();
     TSDataType dataType;
     QueryDataSource queryDataSource;
     try {
       dataType = IoTDB.metaManager.getSeriesType(path.getFullPath());
-      queryDataSource = QueryResourceManager.getInstance().getQueryDataSource(path, context, filter);
-      // update filter by TTL
-      filter = queryDataSource.updateFilterUsingTTL(filter);
+      queryDataSource = QueryResourceManager.getInstance().getQueryDataSource(path, context, valueFilter);
+      // update valueFilter by TTL
+      valueFilter = queryDataSource.updateFilterUsingTTL(valueFilter);
     } catch (Exception e) {
       throw new IOException(e);
     }
 
-    return new SeriesRawDataBatchReader(path, queryPlan.getAllMeasurementsInDevice(path.getDevice()), dataType, context, queryDataSource, null, filter, null);
+    // get the TimeFilter part in SingleSeriesExpression
+    Filter timeFilter = getTimeFilter(valueFilter);
+
+    return new SeriesRawDataBatchReader(path, queryPlan.getAllMeasurementsInDevice(path.getDevice()), dataType, context, queryDataSource, timeFilter, valueFilter, null);
+  }
+
+  /**
+   * extract time filter from a value filter
+   */
+  private Filter getTimeFilter(Filter filter) {
+    if (filter instanceof UnaryFilter && ((UnaryFilter) filter).getFilterType() == FilterType.TIME_FILTER) {
+      return filter;
+    }
+    if (filter instanceof AndFilter) {
+      Filter leftTimeFilter = getTimeFilter(((AndFilter) filter).getLeft());
+      Filter rightTimeFilter = getTimeFilter(((AndFilter) filter).getRight());
+      if (leftTimeFilter != null && rightTimeFilter != null) {
+        return filter;
+      } else if (leftTimeFilter != null) {
+        return leftTimeFilter;
+      } else

Review comment:
       fix




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-iotdb] qiaojialin merged pull request #1651: [IOTDB-849] Optimize value filter query

Posted by GitBox <gi...@apache.org>.
qiaojialin merged pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-iotdb] sonarcloud[bot] commented on pull request #1651: [IOTDB-849] Optimize value filter query

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651#issuecomment-679084155


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/i
 ssues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo.png' alt='No Coverage information' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651) No Coverage information  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-iotdb] sonarcloud[bot] commented on pull request #1651: [IOTDB-849] Optimize value filter query

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on pull request #1651:
URL: https://github.com/apache/incubator-iotdb/pull/1651#issuecomment-680880587


   Kudos, SonarCloud Quality Gate passed!
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug.png' alt='Bug' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=BUG)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability.png' alt='Vulnerability' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=VULNERABILITY) (and [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot.png' alt='Security Hotspot' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/i
 ssues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=SECURITY_HOTSPOT) to review)  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell.png' alt='Code Smell' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL) [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A.png' alt='A' width='16' height='16' />](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache_incubator-iotdb&pullRequest=1651&resolved=false&types=CODE_SMELL)
   
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo.png' alt='No Coverage information' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651) No Coverage information  
   [<img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3.png' alt='0.0%' width='16' height='16' />](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache_incubator-iotdb&pullRequest=1651&metric=new_duplicated_lines_density&view=list)
   
   <img src='https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/message_warning.png' alt='warning' width='16' height='16' /> The version of Java (1.8.0_252) you have used to run this analysis is deprecated and we will stop accepting it from October 2020. Please update to at least Java 11.
   Read more [here](https://sonarcloud.io/documentation/upcoming/)
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org