[GitHub] [hive] deniskuzZ commented on a change in pull request #2266: HIVE-24663 : Batch process in ColStatsProcessor for partitions

[GitBox <gi...@apache.org>]

deniskuzZ commented on a change in pull request #2266:
URL: https://github.com/apache/hive/pull/2266#discussion_r642863679



##########
File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java
##########
@@ -8994,10 +9028,15 @@ public boolean set_aggr_stats_for(SetPartitionsStatsRequest request) throws TExc
             colNames, newStatsMap, request);
       } else { // No merge.
         Table t = getTable(catName, dbName, tableName);
-        for (Map.Entry<String, ColumnStatistics> entry : newStatsMap.entrySet()) {
-          // We don't short-circuit on errors here anymore. That can leave acid stats invalid.
-          ret = updatePartitonColStatsInternal(t, entry.getValue(),
-              request.getValidWriteIdList(), request.getWriteId()) && ret;
+        // We don't short-circuit on errors here anymore. That can leave acid stats invalid.
+        if (newStatsMap.size() > 1) {
+          LOG.info("ETL_PERF started updatePartitionColStatsInBatch");
+          ret = updatePartitionColStatsInBatch(t, newStatsMap,
+                  request.getValidWriteIdList(), request.getWriteId());
+          LOG.info("ETL_PERF done updatePartitionColStatsInBatch");
+        } else {
+          ret = updatePartitonColStatsInternal(t, newStatsMap.values().iterator().next(),

Review comment:
       in future we would need to support both implementations, why can't we generalize? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org