You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "linliu-code (via GitHub)" <gi...@apache.org> on 2024/09/19 23:31:57 UTC

[PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

linliu-code opened a new pull request, #11972:
URL: https://github.com/apache/hudi/pull/11972

   ### Change Logs
   
   1. Update the `HoodieCleanMetadata` with completion time.
   2. Update related logics to make consistent.
   
   ### Impact
   
   Use completion time.
   
   ### Risk level (write none, low medium or high below)
   
   High.
   
   ### Documentation Update
   
   _Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
   
   - _The config description must be updated if new configs are added or the default value of the configs are changed_
   - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
     ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
     changes to the website._
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077][WIP] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362375091

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ac252369f32199eb89a5b68e5cf290137c9a9493 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2364698630

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615",
       "triggerID" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "triggerType" : "PUSH"
     }, {
       "hash" : "b609b2964b5603600fc5b8b4f42e3e2afd042a28",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=645",
       "triggerID" : "b609b2964b5603600fc5b8b4f42e3e2afd042a28",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   * b609b2964b5603600fc5b8b4f42e3e2afd042a28 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=645) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362709618

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615",
       "triggerID" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   * 7046d40ab11eef081b7551f33eb4b9025a716475 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362549727

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ebe7564624d3a0d5741bf7418e31b2e8dd8c3366 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614) 
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   * 7046d40ab11eef081b7551f33eb4b9025a716475 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362545081

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ebe7564624d3a0d5741bf7418e31b2e8dd8c3366 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614) 
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362592703

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "CANCELED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615",
       "triggerID" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ebe7564624d3a0d5741bf7418e31b2e8dd8c3366 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614) 
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   * 7046d40ab11eef081b7551f33eb4b9025a716475 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362538280

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ac252369f32199eb89a5b68e5cf290137c9a9493 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607) 
   * ebe7564624d3a0d5741bf7418e31b2e8dd8c3366 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077][WIP] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362512766

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ac252369f32199eb89a5b68e5cf290137c9a9493 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2364557683

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615",
       "triggerID" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "triggerType" : "PUSH"
     }, {
       "hash" : "b609b2964b5603600fc5b8b4f42e3e2afd042a28",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "b609b2964b5603600fc5b8b4f42e3e2afd042a28",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   * 7046d40ab11eef081b7551f33eb4b9025a716475 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615) 
   * b609b2964b5603600fc5b8b4f42e3e2afd042a28 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "codope (via GitHub)" <gi...@apache.org>.
codope commented on code in PR #11972:
URL: https://github.com/apache/hudi/pull/11972#discussion_r1767911144


##########
hudi-common/src/main/java/org/apache/hudi/common/util/CleanerUtils.java:
##########
@@ -85,8 +85,9 @@ public static HoodieCleanMetadata convertCleanMetadata(String startCleanTime,
         partitionBootstrapMetadataMap.put(stat.getPartitionPath(), bootstrapMetadata);
       }
       totalDeleted += stat.getSuccessDeleteFiles().size();
-      if (earliestCommitToRetain == null) {
-        // This will be the same for all partitions
+
+      // Use the largest completion time.
+      if (needUpdate(stat.getEarliestCommitToRetain(), earliestCommitToRetain)) {
         earliestCommitToRetain = stat.getEarliestCommitToRetain();
         lastCompletedCommitTimestamp = stat.getLastCompletedCommitTimestamp();

Review Comment:
   yes i think we should use completion time for both



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077][WIP] Update the completion time for clean metadata [hudi]

Posted by "linliu-code (via GitHub)" <gi...@apache.org>.
linliu-code commented on code in PR #11972:
URL: https://github.com/apache/hudi/pull/11972#discussion_r1767706053


##########
hudi-common/src/main/java/org/apache/hudi/common/util/CleanerUtils.java:
##########
@@ -85,8 +85,9 @@ public static HoodieCleanMetadata convertCleanMetadata(String startCleanTime,
         partitionBootstrapMetadataMap.put(stat.getPartitionPath(), bootstrapMetadata);
       }
       totalDeleted += stat.getSuccessDeleteFiles().size();
-      if (earliestCommitToRetain == null) {
-        // This will be the same for all partitions
+
+      // Use the largest completion time.
+      if (needUpdate(stat.getEarliestCommitToRetain(), earliestCommitToRetain)) {
         earliestCommitToRetain = stat.getEarliestCommitToRetain();
         lastCompletedCommitTimestamp = stat.getLastCompletedCommitTimestamp();

Review Comment:
   Should we use `stat.getLastCompletedCommitTimestamp()` for both variables?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077][WIP] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362373807

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ac252369f32199eb89a5b68e5cf290137c9a9493 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362543880

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ac252369f32199eb89a5b68e5cf290137c9a9493 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607) 
   * ebe7564624d3a0d5741bf7418e31b2e8dd8c3366 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614) 
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2362536203

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ac252369f32199eb89a5b68e5cf290137c9a9493 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607) 
   * ebe7564624d3a0d5741bf7418e31b2e8dd8c3366 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-8077] Update the completion time for clean metadata [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11972:
URL: https://github.com/apache/hudi/pull/11972#issuecomment-2364600616

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=607",
       "triggerID" : "ac252369f32199eb89a5b68e5cf290137c9a9493",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=614",
       "triggerID" : "ebe7564624d3a0d5741bf7418e31b2e8dd8c3366",
       "triggerType" : "PUSH"
     }, {
       "hash" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "ffc783e315aa82e998a57c6c601e937cb002c368",
       "triggerType" : "PUSH"
     }, {
       "hash" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615",
       "triggerID" : "7046d40ab11eef081b7551f33eb4b9025a716475",
       "triggerType" : "PUSH"
     }, {
       "hash" : "b609b2964b5603600fc5b8b4f42e3e2afd042a28",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=645",
       "triggerID" : "b609b2964b5603600fc5b8b4f42e3e2afd042a28",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * ffc783e315aa82e998a57c6c601e937cb002c368 UNKNOWN
   * 7046d40ab11eef081b7551f33eb4b9025a716475 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=615) 
   * b609b2964b5603600fc5b8b4f42e3e2afd042a28 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=645) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org