You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/02/18 20:39:33 UTC
[GitHub] [hudi] nsivabalan commented on a change in pull request #4821: [HUDI-3435] Do not throw exception when instant to rollback does not …
nsivabalan commented on a change in pull request #4821:
URL: https://github.com/apache/hudi/pull/4821#discussion_r810325230
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java
##########
@@ -429,25 +428,21 @@ public void mergeArchiveFiles(List<FileStatus> compactCandidate) throws IOExcept
.collect(Collectors.groupingBy(i -> Pair.of(i.getTimestamp(),
HoodieInstant.getComparableAction(i.getAction()))));
- // If metadata table is enabled, do not archive instants which are more recent than the last compaction on the
- // metadata table.
- if (config.isMetadataTableEnabled()) {
- try (HoodieTableMetadata tableMetadata = HoodieTableMetadata.create(table.getContext(), config.getMetadataConfig(),
- config.getBasePath(), FileSystemViewStorageConfig.SPILLABLE_DIR.defaultValue())) {
- Option<String> latestCompactionTime = tableMetadata.getLatestCompactionTime();
Review comment:
With the current metadata table design, we validate the deltacommits to read for metadata table based on which completed commits are present in the dataset. So dataset should never be archiving the instants before compaction on metadata table.
##########
File path: hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java
##########
@@ -429,25 +428,21 @@ public void mergeArchiveFiles(List<FileStatus> compactCandidate) throws IOExcept
.collect(Collectors.groupingBy(i -> Pair.of(i.getTimestamp(),
HoodieInstant.getComparableAction(i.getAction()))));
- // If metadata table is enabled, do not archive instants which are more recent than the last compaction on the
- // metadata table.
- if (config.isMetadataTableEnabled()) {
- try (HoodieTableMetadata tableMetadata = HoodieTableMetadata.create(table.getContext(), config.getMetadataConfig(),
- config.getBasePath(), FileSystemViewStorageConfig.SPILLABLE_DIR.defaultValue())) {
- Option<String> latestCompactionTime = tableMetadata.getLatestCompactionTime();
Review comment:
yes.
CC @prashantwason
We have documented the reasoning here https://issues.apache.org/jira/browse/HUDI-2458
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org