[GitHub] [incubator-pinot] siddharthteotia commented on a change in pull request #6288: Use sorted index based filtering only for dictionary encoded column

[GitBox <gi...@apache.org>]

siddharthteotia commented on a change in pull request #6288:
URL: https://github.com/apache/incubator-pinot/pull/6288#discussion_r532849542



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/FilterOperatorUtils.java
##########
@@ -47,9 +47,15 @@ public static BaseFilterOperator getLeafFilterOperator(PredicateEvaluator predic
       return new MatchAllFilterOperator(numDocs);
     }
 
+    // Currently sorted index based filtering is supported only for
+    // dictionary encoded columns. The on-disk segment metadata
+    // will indicate if the column is sorted or not regardless of
+    // whether it is raw or dictionary encoded. Here when creating
+    // the filter operator, we need to make sure that sort filter
+    // operator is used only if the column is sorted and has dictionary.
     Predicate.Type predicateType = predicateEvaluator.getPredicateType();
     if (predicateType == Predicate.Type.RANGE) {
-      if (dataSource.getDataSourceMetadata().isSorted()) {
+      if (dataSource.getDataSourceMetadata().isSorted() && (dataSource.getDictionary() != null)) {

Review comment:
       done

##########
File path: pinot-core/src/main/java/org/apache/pinot/core/operator/filter/FilterOperatorUtils.java
##########
@@ -59,7 +65,7 @@ public static BaseFilterOperator getLeafFilterOperator(PredicateEvaluator predic
     } else if (predicateType == Predicate.Type.REGEXP_LIKE) {
       return new ScanBasedFilterOperator(predicateEvaluator, dataSource, numDocs);
     } else {
-      if (dataSource.getDataSourceMetadata().isSorted()) {
+      if (dataSource.getDataSourceMetadata().isSorted() && (dataSource.getDictionary() != null)) {

Review comment:
       done




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org