[GitHub] [druid] ccaominh commented on a change in pull request #9278: Speed up joins on indexed tables with string keys

[GitBox <gi...@apache.org>]

ccaominh commented on a change in pull request #9278: Speed up joins on indexed tables with string keys
URL: https://github.com/apache/druid/pull/9278#discussion_r373751136
 
 

 ##########
 File path: processing/src/main/java/org/apache/druid/segment/join/table/IndexedTableJoinMatcher.java
 ##########
 @@ -240,13 +243,36 @@ private void advanceCurrentRow()
    */
   private static class ConditionMatcherFactory implements ColumnProcessorFactory<Supplier<IntIterator>>
   {
+    private static final int MAX_NUM_CACHE = 10;
+    private static final int CACHE_MAX_SIZE = 1000;
+
     private final ValueType keyType;
     private final IndexedTable.Index index;
 
+    // DimensionSelector -> (int) dimension id -> (IntList) row numbers
+    private final LoadingCache<DimensionSelector, LoadingCache<Integer, IntList>> dimensionCaches;
+
     ConditionMatcherFactory(ValueType keyType, IndexedTable.Index index)
     {
       this.keyType = keyType;
       this.index = index;
+
+      this.dimensionCaches =
+          Caffeine.newBuilder()
+                  .maximumSize(MAX_NUM_CACHE)
+                  .build(
+                      selector ->
+                          Caffeine.newBuilder()
 
 Review comment:
   On `JoinAndLookupBenchmark.joinIndexedTableStringKey`, using the `LruEvalCache` approach is about 10% faster than using `Caffeine`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org