Re: [PR] Optimize initialization time and memory usage of dimension table [pinot]

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #15130:
URL: https://github.com/apache/pinot/pull/15130#discussion_r1984387023


##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/offline/DimensionTableDataManager.java:
##########
@@ -110,11 +127,19 @@ protected void doInit() {
     }
 
     if (_disablePreload) {
+      Object2LongOpenCustomHashMap<Object[]> lookupTable = new Object2LongOpenCustomHashMap<>(HASH_STRATEGY);

Review Comment:
   The open map is very slow when rehashing because it re-compute hash code for all values. Java `HashMap` caches hash code instead. Can you try benchmarking it with more values and see if the performance gain still exists?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org