[GitHub] [hive] dengzhhu653 commented on a change in pull request #2473: HIVE-25192: Nullify summary for non-native tables

[GitBox <gi...@apache.org>]

dengzhhu653 commented on a change in pull request #2473:
URL: https://github.com/apache/hive/pull/2473#discussion_r675430051



##########
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
##########
@@ -2572,18 +2569,24 @@ public static ContentSummary getInputSummary(final Context ctx, MapWork work, Pa
 
       int numExecutors = getMaxExecutorsForInputListing(ctx.getConf(), pathNeedProcess.size());
       if (numExecutors > 1) {

Review comment:
       The default value of `numExecutors` is 1 unless `pathNeedProcess.size()` > 1 and `hive.exec.input.listing.max.threads` sets to the value bigger than 1. We do not need the global lock when `numExecutors = 1`,  and can be converted to a finer-grained lock. In our cluster, we found that many handler threads were blocked by the global lock when one of federation namespaces failover.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org