[GitHub] [druid] clintropolis commented on a change in pull request #12291: add topn heap optimization when string is dictionary encoded, but not uniquely

[GitBox <gi...@apache.org>]

clintropolis commented on a change in pull request #12291:
URL: https://github.com/apache/druid/pull/12291#discussion_r816448406



##########
File path: processing/src/main/java/org/apache/druid/query/topn/types/StringTopNColumnAggregatesProcessor.java
##########
@@ -172,6 +180,54 @@ private long scanAndAggregateWithCardinalityKnown(
     return processedRows;
   }
 
+  /**
+   * this method uses hash table to store aggregate values ({@link #aggregatesStore}) and is geared towards selectors
+   * which have a dictionary, but those dictionary ids do not necessarily uniquely map to the value of
+   * {@link DimensionSelector#lookupName(int)}. This method keeps a cache of dictionary id to aggregators, which it uses
+   * to skip name lookup if that id has already been encountered, avoiding decoding utf8 bytes, or worse, recomputing
+   * an expression virtual column or some lookup value.
+   *
+   */
+  private long scanAndAggregateWithDictionary(
+      TopNQuery query,
+      Cursor cursor,
+      DimensionSelector selector
+  )
+  {
+    Int2ObjectOpenHashMap<Aggregator[]> aggsCache = new Int2ObjectOpenHashMap<>();

Review comment:
       yeah, that should work :+1:




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org