You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by "hofi1 (via GitHub)" <gi...@apache.org> on 2023/10/17 23:54:21 UTC

[PR] Fix flaky tests (#2645) [helix]

hofi1 opened a new pull request, #2666:
URL: https://github.com/apache/helix/pull/2666

   ## Issues
   
   - [x] My PR addresses the following Helix issues and references them in the PR description:
   
   Fixes #2645
   
   ## Description
   
   Sets return the elements in a non-deterministic order, which means that this assertion is not correct, because it checks whether the collections contain the same elements in the same order. This leads to a flack test. To fix this problem, the assertion has been rewritten to check if the collections contain the same amount of elements as well as booth collections contain all values of the other collection.
   
   Tests that cause problems: 
   https://github.com/hofi1/helix/blob/6241bcbf5f9fab1726c87c20b5e948daa95b7bbf/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java#L133-L141
   
   and 
   https://github.com/hofi1/helix/blob/6241bcbf5f9fab1726c87c20b5e948daa95b7bbf/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java#L144-L152
   
   This problem was found by the [NonDex](https://github.com/TestingResearchIllinois/NonDex) Engine – to reproduce run
   ```shell
   mvn -pl helix-rest edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.helix.rest.metadatastore.TestZkMetadataStoreDirectory
   ``` 
   
   ## Solution
   Replace the assertions with a custom matcher, which checks if the sets contain the same elements without taking care of the order of the elements returned by the set. 
   
   Custom Matcher:
   https://github.com/hofi1/helix/blob/9cd4181fe361ee2015e3e8fd507e158a2eb78991/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java#L378-L383
   
   ## Tests
   
   No test have been written – one existing test has been updated.
   
   - The following is the result of the "mvn test" command on the appropriate module:
   
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD SUCCESS
   [INFO] ------------------------------------------------------------------------
   [INFO] Total time:  02:51 min
   [INFO] Finished at: 2023-10-04T23:16:04-05:00
   [INFO] ------------------------------------------------------------------------
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Fix flaky tests org.apache.helix.rest.metadatastore.TestZkMetadataStoreDirectory (#2645) [helix]

Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue merged PR #2666:
URL: https://github.com/apache/helix/pull/2666


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Fix flaky tests org.apache.helix.rest.metadatastore.TestZkMetadataStoreDirectory (#2645) [helix]

Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on code in PR #2666:
URL: https://github.com/apache/helix/pull/2666#discussion_r1500072553


##########
helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java:
##########
@@ -375,4 +375,10 @@ private void clearRoutingData() throws Exception {
       return true;
     }, TestHelper.WAIT_DURATION), "Routing data path should be deleted after the tests.");
   }
+  private void assertCollectionsContainSameElementsIgnoringOrder(Collection<String> collection1,
+   Collection<String> collection2) {
+     Assert.assertEquals(collection2.size(), collection1.size());
+     Assert.assertTrue(collection2.containsAll(collection1));
+     Assert.assertTrue(collection1.containsAll(collection2));
+  }

Review Comment:
   Would be good to have it in the abstractestclass to be a common used method.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org


Re: [PR] Fix flaky tests org.apache.helix.rest.metadatastore.TestZkMetadataStoreDirectory (#2645) [helix]

Posted by "junkaixue (via GitHub)" <gi...@apache.org>.
junkaixue commented on PR #2666:
URL: https://github.com/apache/helix/pull/2666#issuecomment-1967329852

   Since no update for a couple of days. I think it is OK to just fix this and move forward.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org