You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "codope (via GitHub)" <gi...@apache.org> on 2024/05/01 01:19:14 UTC
Re: [PR] [HUDI-7144] Build storage partition stats index and use it for data skipping [hudi]
codope commented on code in PR #10352:
URL: https://github.com/apache/hudi/pull/10352#discussion_r1585773581
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieTableMetadataUtil.java:
##########
@@ -1872,4 +1883,175 @@ public HoodieRecord next() {
}
};
}
+
+ public static HoodieData<HoodieRecord> convertFilesToPartitionStatsRecords(HoodieEngineContext engineContext,
+ List<DirectoryInfo> partitionInfoList,
Review Comment:
Yes, this is just for initialization. We just log a warning and continue for other filegroups. This is what colstats is also doing. However, given that partition stats will be enabled by default, I think it makes sense to throw exception. Will do it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org