You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/09/14 08:04:41 UTC

[GitHub] [hudi] voonhous opened a new pull request, #6669: [HUDI-4841] Fix sort idempotency issue

voonhous opened a new pull request, #6669:
URL: https://github.com/apache/hudi/pull/6669

   ### Change Logs
   
   BlockLocation sort compare implementation is not idempotent, causing `getBlockIndexForPosition()` to throw `IllegalArgumentException`.
   
   ### Impact
   
   _Describe any public API or user-facing feature change or any performance impact._
   
   **Risk level: none | low | medium | high**
   
   _Choose one. If medium or high, explain what verification was done to mitigate the risks._
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [ ] Change Logs and Impact were stated clearly
   - [ ] Adequate tests were added if applicable
   - [ ] CI passed
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix sort idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1246455821

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5ccbb91fecf43acc0ef6326e83fded2a58039d86 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249018925

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     }, {
       "hash" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417",
       "triggerID" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5ccbb91fecf43acc0ef6326e83fded2a58039d86 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354) 
   * c9221a1b17c255166f9562bd850390384f9d0fa2 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] voonhous commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
voonhous commented on code in PR #6669:
URL: https://github.com/apache/hudi/pull/6669#discussion_r972661367


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java:
##########
@@ -214,13 +213,7 @@ public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {
 
         // get the block locations and make sure they are in order with respect to their offset
         final BlockLocation[] blocks = fs.getFileBlockLocations(file, 0, len);
-        Arrays.sort(blocks, new Comparator<BlockLocation>() {
-          @Override
-          public int compare(BlockLocation o1, BlockLocation o2) {
-            long diff = o1.getLength() - o2.getOffset();
-            return Long.compare(diff, 0L);
-          }
-        });
+        Arrays.sort(blocks, CopyOnWriteInputFormat::compareBlockLocations);
 

Review Comment:
   Added the change



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249014117

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     }, {
       "hash" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5ccbb91fecf43acc0ef6326e83fded2a58039d86 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354) 
   * c9221a1b17c255166f9562bd850390384f9d0fa2 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249958394

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     }, {
       "hash" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417",
       "triggerID" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "71859eef948188566ff065ebcfe1a8da92f5109e",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "71859eef948188566ff065ebcfe1a8da92f5109e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * c9221a1b17c255166f9562bd850390384f9d0fa2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417) 
   * 71859eef948188566ff065ebcfe1a8da92f5109e UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix sort idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1247083707

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5ccbb91fecf43acc0ef6326e83fded2a58039d86 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249670410

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     }, {
       "hash" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417",
       "triggerID" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * c9221a1b17c255166f9562bd850390384f9d0fa2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] danny0405 commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
danny0405 commented on code in PR #6669:
URL: https://github.com/apache/hudi/pull/6669#discussion_r972658464


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java:
##########
@@ -214,13 +213,7 @@ public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {
 
         // get the block locations and make sure they are in order with respect to their offset
         final BlockLocation[] blocks = fs.getFileBlockLocations(file, 0, len);
-        Arrays.sort(blocks, new Comparator<BlockLocation>() {
-          @Override
-          public int compare(BlockLocation o1, BlockLocation o2) {
-            long diff = o1.getLength() - o2.getOffset();
-            return Long.compare(diff, 0L);
-          }
-        });
+        Arrays.sort(blocks, CopyOnWriteInputFormat::compareBlockLocations);
 

Review Comment:
   Does `Arrays.sort(blocks, Comparator.comparingLong(BlockLocation::getOffset));` work here ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] voonhous commented on pull request #6669: [HUDI-4841] Fix sort idempotency issue

Posted by GitBox <gi...@apache.org>.
voonhous commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1247146902

   This bug mainly affects reading


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] voonhous commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
voonhous commented on code in PR #6669:
URL: https://github.com/apache/hudi/pull/6669#discussion_r972658947


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java:
##########
@@ -214,13 +213,7 @@ public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {
 
         // get the block locations and make sure they are in order with respect to their offset
         final BlockLocation[] blocks = fs.getFileBlockLocations(file, 0, len);
-        Arrays.sort(blocks, new Comparator<BlockLocation>() {
-          @Override
-          public int compare(BlockLocation o1, BlockLocation o2) {
-            long diff = o1.getLength() - o2.getOffset();
-            return Long.compare(diff, 0L);
-          }
-        });
+        Arrays.sort(blocks, CopyOnWriteInputFormat::compareBlockLocations);
 

Review Comment:
   Yes, should work fine here. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1249960054

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "DELETED",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     }, {
       "hash" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417",
       "triggerID" : "c9221a1b17c255166f9562bd850390384f9d0fa2",
       "triggerType" : "PUSH"
     }, {
       "hash" : "71859eef948188566ff065ebcfe1a8da92f5109e",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11437",
       "triggerID" : "71859eef948188566ff065ebcfe1a8da92f5109e",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * c9221a1b17c255166f9562bd850390384f9d0fa2 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11417) 
   * 71859eef948188566ff065ebcfe1a8da92f5109e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11437) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] voonhous commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
voonhous commented on code in PR #6669:
URL: https://github.com/apache/hudi/pull/6669#discussion_r973513763


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java:
##########
@@ -214,13 +213,7 @@ public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {
 
         // get the block locations and make sure they are in order with respect to their offset
         final BlockLocation[] blocks = fs.getFileBlockLocations(file, 0, len);
-        Arrays.sort(blocks, new Comparator<BlockLocation>() {
-          @Override
-          public int compare(BlockLocation o1, BlockLocation o2) {
-            long diff = o1.getLength() - o2.getOffset();
-            return Long.compare(diff, 0L);
-          }
-        });
+        Arrays.sort(blocks, CopyOnWriteInputFormat::compareBlockLocations);
 

Review Comment:
   My bad, i modified it to `comparing` instead of `comparingLong`. 
   
   I have added the required changes. Apologies.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] yihua commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6669:
URL: https://github.com/apache/hudi/pull/6669#discussion_r973522477


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java:
##########
@@ -214,13 +213,7 @@ public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {
 
         // get the block locations and make sure they are in order with respect to their offset
         final BlockLocation[] blocks = fs.getFileBlockLocations(file, 0, len);
-        Arrays.sort(blocks, new Comparator<BlockLocation>() {
-          @Override
-          public int compare(BlockLocation o1, BlockLocation o2) {
-            long diff = o1.getLength() - o2.getOffset();
-            return Long.compare(diff, 0L);
-          }
-        });
+        Arrays.sort(blocks, CopyOnWriteInputFormat::compareBlockLocations);
 

Review Comment:
   No worries.  I'll merge this PR once CI passes.  Thanks for the fix!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] hudi-bot commented on pull request #6669: [HUDI-4841] Fix sort idempotency issue

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #6669:
URL: https://github.com/apache/hudi/pull/6669#issuecomment-1246462091

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354",
       "triggerID" : "5ccbb91fecf43acc0ef6326e83fded2a58039d86",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 5ccbb91fecf43acc0ef6326e83fded2a58039d86 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=11354) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] yihua commented on a diff in pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #6669:
URL: https://github.com/apache/hudi/pull/6669#discussion_r973507946


##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cow/CopyOnWriteInputFormat.java:
##########
@@ -214,13 +213,7 @@ public FileInputSplit[] createInputSplits(int minNumSplits) throws IOException {
 
         // get the block locations and make sure they are in order with respect to their offset
         final BlockLocation[] blocks = fs.getFileBlockLocations(file, 0, len);
-        Arrays.sort(blocks, new Comparator<BlockLocation>() {
-          @Override
-          public int compare(BlockLocation o1, BlockLocation o2) {
-            long diff = o1.getLength() - o2.getOffset();
-            return Long.compare(diff, 0L);
-          }
-        });
+        Arrays.sort(blocks, CopyOnWriteInputFormat::compareBlockLocations);
 

Review Comment:
   @voonhous I don't see the change of using `comparingLong` in the latest commit.  Have you pushed your changes?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [hudi] yihua merged pull request #6669: [HUDI-4841] Fix BlockLocation array sorting idempotency issue

Posted by GitBox <gi...@apache.org>.
yihua merged PR #6669:
URL: https://github.com/apache/hudi/pull/6669


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org