You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "the-other-tim-brown (via GitHub)" <gi...@apache.org> on 2024/04/12 12:57:07 UTC

[PR] [MINOR] Make ordering deterministic in small file selection [hudi]

the-other-tim-brown opened a new pull request, #11008:
URL: https://github.com/apache/hudi/pull/11008

   ### Change Logs
   
   Makes the ordering deterministic to get consistent results and avoid any issues in tests
   
   ### Impact
   
   Small file selection is consistent (mostly helps tests be consistent)
   
   ### Risk level (write none, low medium or high below)
   
   None
   
   ### Documentation Update
   
   _Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
   
   - _The config description must be updated if new configs are added or the default value of the configs are changed_
   - _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
     ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
     changes to the website._
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2077813217

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     }, {
       "hash" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490",
       "triggerID" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 17c9f852515b869c08d317553fac266ff493a3f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2051806863

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dde68f9c2bda3e1045d3bcda6c2472072395a0 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #11008:
URL: https://github.com/apache/hudi/pull/11008#discussion_r1571650091


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/deltacommit/SparkUpsertDeltaCommitPartitioner.java:
##########
@@ -89,10 +89,13 @@ protected List<SmallFile> getSmallFiles(String partitionPath) {
   private List<FileSlice> getSmallFileCandidates(String partitionPath, HoodieInstant latestCommitInstant) {
     // If we can index log files, we can add more inserts to log files for fileIds NOT including those under
     // pending compaction
+    Comparator<FileSlice> comparator = Comparator.<FileSlice, Long>comparing(fileSlice -> getTotalFileSize(fileSlice))
+        .thenComparing(FileSlice::getFileId);
     if (table.getIndex().canIndexLogFiles()) {
       return table.getSliceView()
               .getLatestFileSlicesBeforeOrOn(partitionPath, latestCommitInstant.getTimestamp(), false)
               .filter(this::isSmallFile)
+              .sorted(comparator)
               .collect(Collectors.toList());

Review Comment:
   I've been trying but do not understand why these tests are coupled to small file handling. I think that the approach to testing is a bit strange to rely on that feature to test something in delta streamer for example.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2061508850

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 08eee17c0e936c02e100b65aeba27f81a232452c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231) 
   * 4b93a6017bece2bbe7233d316912e322e77f7d49 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2065666227

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 82029e70eec8c77e1c64bf9f751200c6962777ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351) 
   * 6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2065467861

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e5a2713d07581824214bcc7b9321e3d1cb371c02 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322) 
   * 82029e70eec8c77e1c64bf9f751200c6962777ec UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #11008:
URL: https://github.com/apache/hudi/pull/11008#discussion_r1563428947


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/deltacommit/SparkUpsertDeltaCommitPartitioner.java:
##########
@@ -89,10 +89,13 @@ protected List<SmallFile> getSmallFiles(String partitionPath) {
   private List<FileSlice> getSmallFileCandidates(String partitionPath, HoodieInstant latestCommitInstant) {
     // If we can index log files, we can add more inserts to log files for fileIds NOT including those under
     // pending compaction
+    Comparator<FileSlice> comparator = Comparator.<FileSlice, Long>comparing(fileSlice -> getTotalFileSize(fileSlice))
+        .thenComparing(FileSlice::getFileId);
     if (table.getIndex().canIndexLogFiles()) {
       return table.getSliceView()
               .getLatestFileSlicesBeforeOrOn(partitionPath, latestCommitInstant.getTimestamp(), false)
               .filter(this::isSmallFile)
+              .sorted(comparator)
               .collect(Collectors.toList());

Review Comment:
   I think it makes sense to prefer the smallest files first as candidates to minimize IO



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11008:
URL: https://github.com/apache/hudi/pull/11008#discussion_r1563427282


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/deltacommit/SparkUpsertDeltaCommitPartitioner.java:
##########
@@ -89,10 +89,13 @@ protected List<SmallFile> getSmallFiles(String partitionPath) {
   private List<FileSlice> getSmallFileCandidates(String partitionPath, HoodieInstant latestCommitInstant) {
     // If we can index log files, we can add more inserts to log files for fileIds NOT including those under
     // pending compaction
+    Comparator<FileSlice> comparator = Comparator.<FileSlice, Long>comparing(fileSlice -> getTotalFileSize(fileSlice))
+        .thenComparing(FileSlice::getFileId);
     if (table.getIndex().canIndexLogFiles()) {
       return table.getSliceView()
               .getLatestFileSlicesBeforeOrOn(partitionPath, latestCommitInstant.getTimestamp(), false)
               .filter(this::isSmallFile)
+              .sorted(comparator)
               .collect(Collectors.toList());

Review Comment:
   Should we just fix the tests? Do we have gains for the sort in production?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2065630105

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 82029e70eec8c77e1c64bf9f751200c6962777ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351) 
   * 6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11008:
URL: https://github.com/apache/hudi/pull/11008#discussion_r1569737390


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/deltacommit/SparkUpsertDeltaCommitPartitioner.java:
##########
@@ -89,10 +89,13 @@ protected List<SmallFile> getSmallFiles(String partitionPath) {
   private List<FileSlice> getSmallFileCandidates(String partitionPath, HoodieInstant latestCommitInstant) {
     // If we can index log files, we can add more inserts to log files for fileIds NOT including those under
     // pending compaction
+    Comparator<FileSlice> comparator = Comparator.<FileSlice, Long>comparing(fileSlice -> getTotalFileSize(fileSlice))
+        .thenComparing(FileSlice::getFileId);
     if (table.getIndex().canIndexLogFiles()) {
       return table.getSliceView()
               .getLatestFileSlicesBeforeOrOn(partitionPath, latestCommitInstant.getTimestamp(), false)
               .filter(this::isSmallFile)
+              .sorted(comparator)
               .collect(Collectors.toList());

Review Comment:
   Fine, just fix all the test failures and let's see what use cases are affected.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #11008:
URL: https://github.com/apache/hudi/pull/11008#discussion_r1571644343


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/deltacommit/SparkUpsertDeltaCommitPartitioner.java:
##########
@@ -89,10 +89,13 @@ protected List<SmallFile> getSmallFiles(String partitionPath) {
   private List<FileSlice> getSmallFileCandidates(String partitionPath, HoodieInstant latestCommitInstant) {
     // If we can index log files, we can add more inserts to log files for fileIds NOT including those under
     // pending compaction
+    Comparator<FileSlice> comparator = Comparator.<FileSlice, Long>comparing(fileSlice -> getTotalFileSize(fileSlice))
+        .thenComparing(FileSlice::getFileId);
     if (table.getIndex().canIndexLogFiles()) {
       return table.getSliceView()
               .getLatestFileSlicesBeforeOrOn(partitionPath, latestCommitInstant.getTimestamp(), false)
               .filter(this::isSmallFile)
+              .sorted(comparator)
               .collect(Collectors.toList());

Review Comment:
   Looks like the test failure takes some time to fix, is it easy to make the tests deterministic?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2061488971

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 08eee17c0e936c02e100b65aeba27f81a232452c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231) 
   * 4b93a6017bece2bbe7233d316912e322e77f7d49 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2065473833

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e5a2713d07581824214bcc7b9321e3d1cb371c02 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322) 
   * 82029e70eec8c77e1c64bf9f751200c6962777ec Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052104198

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2080026236

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     }, {
       "hash" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490",
       "triggerID" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3204af717737f8a4f75d1534b2ec4415ebd08676",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "3204af717737f8a4f75d1534b2ec4415ebd08676",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 17c9f852515b869c08d317553fac266ff493a3f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490) 
   * 3204af717737f8a4f75d1534b2ec4415ebd08676 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2061782916

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e5a2713d07581824214bcc7b9321e3d1cb371c02 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2077725887

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     }, {
       "hash" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490",
       "triggerID" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355) 
   * 17c9f852515b869c08d317553fac266ff493a3f2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2077709142

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     }, {
       "hash" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355) 
   * 17c9f852515b869c08d317553fac266ff493a3f2 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052737016

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 08eee17c0e936c02e100b65aeba27f81a232452c Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2080105462

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     }, {
       "hash" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490",
       "triggerID" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3204af717737f8a4f75d1534b2ec4415ebd08676",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23521",
       "triggerID" : "3204af717737f8a4f75d1534b2ec4415ebd08676",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 3204af717737f8a4f75d1534b2ec4415ebd08676 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23521) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052662120

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220) 
   * 08eee17c0e936c02e100b65aeba27f81a232452c Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #11008:
URL: https://github.com/apache/hudi/pull/11008#discussion_r1571633643


##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/deltacommit/SparkUpsertDeltaCommitPartitioner.java:
##########
@@ -89,10 +89,13 @@ protected List<SmallFile> getSmallFiles(String partitionPath) {
   private List<FileSlice> getSmallFileCandidates(String partitionPath, HoodieInstant latestCommitInstant) {
     // If we can index log files, we can add more inserts to log files for fileIds NOT including those under
     // pending compaction
+    Comparator<FileSlice> comparator = Comparator.<FileSlice, Long>comparing(fileSlice -> getTotalFileSize(fileSlice))
+        .thenComparing(FileSlice::getFileId);
     if (table.getIndex().canIndexLogFiles()) {
       return table.getSliceView()
               .getLatestFileSlicesBeforeOrOn(partitionPath, latestCommitInstant.getTimestamp(), false)
               .filter(this::isSmallFile)
+              .sorted(comparator)
               .collect(Collectors.toList());

Review Comment:
   @danny0405 the test failing is flakey. It is try to test some spark exception but it is non-deterministic. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2051912011

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dde68f9c2bda3e1045d3bcda6c2472072395a0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218) 
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2051898070

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dde68f9c2bda3e1045d3bcda6c2472072395a0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218) 
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2080037055

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     }, {
       "hash" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490",
       "triggerID" : "17c9f852515b869c08d317553fac266ff493a3f2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "3204af717737f8a4f75d1534b2ec4415ebd08676",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23521",
       "triggerID" : "3204af717737f8a4f75d1534b2ec4415ebd08676",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 17c9f852515b869c08d317553fac266ff493a3f2 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23490) 
   * 3204af717737f8a4f75d1534b2ec4415ebd08676 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23521) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052420166

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052654450

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220) 
   * 08eee17c0e936c02e100b65aeba27f81a232452c UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2051793682

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dde68f9c2bda3e1045d3bcda6c2472072395a0 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2065711149

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     }, {
       "hash" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355",
       "triggerID" : "6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 6d5468f8ee2c27e1178dd6cb6c6cacd2d965b136 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23355) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2061681806

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4b93a6017bece2bbe7233d316912e322e77f7d49 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319) 
   * e5a2713d07581824214bcc7b9321e3d1cb371c02 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous closed pull request #11008: [MINOR] Make ordering deterministic in small file selection
URL: https://github.com/apache/hudi/pull/11008


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2061658456

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 4b93a6017bece2bbe7233d316912e322e77f7d49 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319) 
   * e5a2713d07581824214bcc7b9321e3d1cb371c02 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2065517817

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     }, {
       "hash" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23231",
       "triggerID" : "08eee17c0e936c02e100b65aeba27f81a232452c",
       "triggerType" : "PUSH"
     }, {
       "hash" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23319",
       "triggerID" : "4b93a6017bece2bbe7233d316912e322e77f7d49",
       "triggerType" : "PUSH"
     }, {
       "hash" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23322",
       "triggerID" : "e5a2713d07581824214bcc7b9321e3d1cb371c02",
       "triggerType" : "PUSH"
     }, {
       "hash" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351",
       "triggerID" : "82029e70eec8c77e1c64bf9f751200c6962777ec",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 82029e70eec8c77e1c64bf9f751200c6962777ec Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23351) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2051884870

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * e7dde68f9c2bda3e1045d3bcda6c2472072395a0 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052311371

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-2052326296

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23218",
       "triggerID" : "e7dde68f9c2bda3e1045d3bcda6c2472072395a0",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220",
       "triggerID" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "triggerType" : "PUSH"
     }, {
       "hash" : "baaff5d03b4199e0aa188492cfa8a5fe2908a47e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "2052311371",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * baaff5d03b4199e0aa188492cfa8a5fe2908a47e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=23220) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [MINOR] Make ordering deterministic in small file selection [hudi]

Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous commented on PR #11008:
URL: https://github.com/apache/hudi/pull/11008#issuecomment-6700311979

   Temporarily closing this PR while we debug a GitHub settings error on master (#20471); it will be reopened shortly, no action needed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org