[GitHub] [hive] kasakrisz commented on a diff in pull request #4000: HIVE-27007: Iceberg: Use BasicStats from iceberg table's currrentSnap…

["kasakrisz (via GitHub)" <gi...@apache.org>]

kasakrisz commented on code in PR #4000:
URL: https://github.com/apache/hive/pull/4000#discussion_r1091850767


##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -277,23 +277,31 @@ private static Statistics collectStatistics(HiveConf conf, PrunedPartitionList p
     boolean metaTable = table.getMetaTable() != null;
 
     if (!table.isPartitioned()) {
+      long ds, nr, fs;
+      if (table.isNonNative() && table.getStorageHandler().canProvideBasicStatistics()) {
+        Map<String, String> icebergBasicStatMap = table.getStorageHandler().getBasicStatistics(Partish.buildFor(table));
+        ds = Long.parseLong(icebergBasicStatMap.get("totalSize"));
+        nr = Long.parseLong(icebergBasicStatMap.get("numRows"));
+        fs = Long.parseLong(icebergBasicStatMap.get("numFiles"));

Review Comment:
   Should a null check be added here and/or handle NumberFormatException?
   
   What would be the stats values if it can not be extracted via storage handler?
   In case of non-storage handler path there are so called `enhancers` to estimate and fix the values:
   ```
   basicStatsFactory.addEnhancer(new BasicStats.RowNumEstimator(estimateRowSizeFromSchema(conf, schema)));
   basicStatsFactory.addEnhancer(new BasicStats.SetMinRowNumber01());
   ```



##########
ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java:
##########
@@ -277,23 +277,31 @@ private static Statistics collectStatistics(HiveConf conf, PrunedPartitionList p
     boolean metaTable = table.getMetaTable() != null;
 
     if (!table.isPartitioned()) {
+      long ds, nr, fs;

Review Comment:
   Could you please declare each variable in separate lines.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org
For additional commands, e-mail: gitbox-help@hive.apache.org