You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2019/10/06 19:31:32 UTC
[GitHub] [incubator-hudi] bvaradar commented on a change in pull request
#942: [HUDI-137] Fix state transitions for Hudi cleaning action
bvaradar commented on a change in pull request #942: [HUDI-137] Fix state transitions for Hudi cleaning action
URL: https://github.com/apache/incubator-hudi/pull/942#discussion_r331806225
##########
File path: hudi-client/src/main/java/org/apache/hudi/HoodieWriteClient.java
##########
@@ -1026,22 +1107,25 @@ private void clean(String startCleanTime) throws HoodieIOException {
// Create the metadata and save it
HoodieCleanMetadata metadata = AvroUtils
- .convertCleanMetadata(startCleanTime, durationInMs, cleanStats);
+ .convertCleanMetadata(cleanInstant.getTimestamp(), durationInMs, cleanStats);
logger.info("Cleaned " + metadata.getTotalFilesDeleted() + " files");
- metrics
- .updateCleanMetrics(durationInMs.orElseGet(() -> -1L), metadata.getTotalFilesDeleted());
+ metrics.updateCleanMetrics(durationInMs.orElseGet(() -> -1L), metadata.getTotalFilesDeleted());
table.getActiveTimeline()
- .saveAsComplete(new HoodieInstant(true, HoodieTimeline.CLEAN_ACTION, startCleanTime),
- AvroUtils.serializeCleanMetadata(metadata));
- logger.info("Marked clean started on " + startCleanTime + " as complete");
+ .transitionCleanInflightToComplete(new HoodieInstant(true, HoodieTimeline.CLEAN_ACTION,
+ cleanInstant.getTimestamp()), AvroUtils.serializeCleanMetadata(metadata));
+ logger.info("Marked clean started on " + cleanInstant.getTimestamp() + " as complete");
+ /**
+ * @vinoth/@n3nash : I have commented this code as archiving can handle cleaner actions. Is this ok ?
if (!table.getActiveTimeline().getCleanerTimeline().empty()) {
// Cleanup of older cleaner meta files
// TODO - make the commit archival generic and archive clean metadata
FSUtils.deleteOlderCleanMetaFiles(fs, table.getMetaClient().getMetaPath(),
table.getActiveTimeline().getCleanerTimeline().getInstants());
}
+ **/
Review comment:
Will confirm with @n3nash before I remove this part.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services