[GitHub] [hive] rbalamohan commented on a change in pull request #2906: HIVE-24805: Compactor: Initiator shouldn't fetch table details again …

[GitBox <gi...@apache.org>]

rbalamohan commented on a change in pull request #2906:
URL: https://github.com/apache/hive/pull/2906#discussion_r782617149



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java
##########
@@ -164,7 +164,7 @@ public void run() {
 
           for (CompactionInfo ci : potentials) {
             try {
-              Table t = resolveTable(ci);
+              Table t = resolveTableAndCache(ci);

Review comment:
       Issue is more pronounced in the following code path.
   
   {code}
   Set<CompactionInfo> potentials = compactionExecutor.submit(() ->
               txnHandler.findPotentialCompactions(abortedThreshold, abortedTimeThreshold, compactionInterval)
                 .parallelStream()
                 .filter(ci -> isEligibleForCompaction(ci, currentCompactions, skipDBs, skipTables))
                 .collect(Collectors.toSet())).get();
   {code}
   
   i.e in `isEligibleForCompaction`.  
   
   Instead of introducing the cache and invalidating it later, is it possible to create a simple hashmap in the method itself and use it on need basis? (e.g if there are multiple partitions, it will just do one lookup for table and the rest of the partitions will make use of the value in map).




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org