You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xtable.apache.org by "ashvina (via GitHub)" <gi...@apache.org> on 2025/03/06 00:27:37 UTC

[PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

ashvina opened a new pull request, #662:
URL: https://github.com/apache/incubator-xtable/pull/662

   Fixes #658 
   
   This change fixes a test only. The test is currently comparing the equality of two lists, which would fail if the order of elements in the lists differ. Since the test is order insensitive, the right solution is to convert the lists to sets. Set equality is order independent.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

Posted by "ashvina (via GitHub)" <gi...@apache.org>.
ashvina commented on code in PR #662:
URL: https://github.com/apache/incubator-xtable/pull/662#discussion_r1984122322


##########
xtable-core/src/test/java/org/apache/xtable/hudi/TestBaseFileUpdatesExtractor.java:
##########
@@ -135,7 +136,7 @@ void convertDiff() {
     expectedPartitionToReplacedFileIds.put(
         partitionPath1, Arrays.asList(fileName3, fileIdForFile4));
     expectedPartitionToReplacedFileIds.put(partitionPath2, Collections.singletonList(fileName5));
-    assertEquals(
+    assertEqualsIgnoreOrder(
         expectedPartitionToReplacedFileIds, replaceMetadata.getPartitionToReplacedFileIds());

Review Comment:
   Hi @vinishjail97 . I didn't understand. Have you tried this change, does it compile?
   1. `expectedPartitionToReplacedFileIds` is a `HashMap` which cannnot be used to initialize a `HashSet`
   2. The order mismatch happens in the `values` of the HashMaps, where the `values` are `List`s. The tests fails when the values in the lists are out of order. Please see the example in the bug report for reference.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

Posted by "ashvina (via GitHub)" <gi...@apache.org>.
ashvina commented on code in PR #662:
URL: https://github.com/apache/incubator-xtable/pull/662#discussion_r1984122322


##########
xtable-core/src/test/java/org/apache/xtable/hudi/TestBaseFileUpdatesExtractor.java:
##########
@@ -135,7 +136,7 @@ void convertDiff() {
     expectedPartitionToReplacedFileIds.put(
         partitionPath1, Arrays.asList(fileName3, fileIdForFile4));
     expectedPartitionToReplacedFileIds.put(partitionPath2, Collections.singletonList(fileName5));
-    assertEquals(
+    assertEqualsIgnoreOrder(
         expectedPartitionToReplacedFileIds, replaceMetadata.getPartitionToReplacedFileIds());

Review Comment:
   Hi @vinishjail97 . I didn't understand. Have you tried this change, does it compile?
   1. `expectedPartitionToReplacedFileIds` is a `HashMap` which cannnot be used to initialize a `HashSet`
   2. The order mismatch happens in the valueSet of the Hashmap, where the valueSet is actually List values.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #662:
URL: https://github.com/apache/incubator-xtable/pull/662#discussion_r1984112331


##########
xtable-core/src/test/java/org/apache/xtable/hudi/TestBaseFileUpdatesExtractor.java:
##########
@@ -135,7 +136,7 @@ void convertDiff() {
     expectedPartitionToReplacedFileIds.put(
         partitionPath1, Arrays.asList(fileName3, fileIdForFile4));
     expectedPartitionToReplacedFileIds.put(partitionPath2, Collections.singletonList(fileName5));
-    assertEquals(
+    assertEqualsIgnoreOrder(
         expectedPartitionToReplacedFileIds, replaceMetadata.getPartitionToReplacedFileIds());

Review Comment:
   This can be simplified to ? 
   `assertEquals(new HashSet<>(expectedPartitionToReplacedFileIds), new HashSet<>(replaceMetadata.getPartitionToReplacedFileIds())) `



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

Posted by "ashvina (via GitHub)" <gi...@apache.org>.
ashvina merged PR #662:
URL: https://github.com/apache/incubator-xtable/pull/662


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #662:
URL: https://github.com/apache/incubator-xtable/pull/662#discussion_r1984146003


##########
xtable-core/src/test/java/org/apache/xtable/hudi/TestBaseFileUpdatesExtractor.java:
##########
@@ -135,7 +136,7 @@ void convertDiff() {
     expectedPartitionToReplacedFileIds.put(
         partitionPath1, Arrays.asList(fileName3, fileIdForFile4));
     expectedPartitionToReplacedFileIds.put(partitionPath2, Collections.singletonList(fileName5));
-    assertEquals(
+    assertEqualsIgnoreOrder(
         expectedPartitionToReplacedFileIds, replaceMetadata.getPartitionToReplacedFileIds());

Review Comment:
   My bad didn't see it was `Map<String, List<String>>`, change LGTM. 
   
   `Hudi does not depend on the order of the files in the lists?`
   Yes it does not depend on order.   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix flaky test, ignore order of elements in list validation [incubator-xtable]

Posted by "ashvina (via GitHub)" <gi...@apache.org>.
ashvina commented on code in PR #662:
URL: https://github.com/apache/incubator-xtable/pull/662#discussion_r1984125263


##########
xtable-core/src/test/java/org/apache/xtable/hudi/TestBaseFileUpdatesExtractor.java:
##########
@@ -135,7 +136,7 @@ void convertDiff() {
     expectedPartitionToReplacedFileIds.put(
         partitionPath1, Arrays.asList(fileName3, fileIdForFile4));
     expectedPartitionToReplacedFileIds.put(partitionPath2, Collections.singletonList(fileName5));
-    assertEquals(
+    assertEqualsIgnoreOrder(
         expectedPartitionToReplacedFileIds, replaceMetadata.getPartitionToReplacedFileIds());

Review Comment:
   Also, could you please confirm that Hudi does not depend on the order of the files in the lists?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org