[GitHub] [ignite-3] kaiyaok2 opened a new pull request, #888: Fix indeterminate unit test results due to order dependencies

[GitBox <gi...@apache.org>]

kaiyaok2 opened a new pull request, #888:
URL: https://github.com/apache/ignite-3/pull/888

   **Description**
   In the module `cli`, a couple of tests, namely `IgniteCliInterfaceTest$Cluster#initSuccess()` and `CliConfigSubCommandTest#noKey()`, can fail nondeterministically when been run in different JVMs. The issue can be found by running command `mvn edu.illinois:nondex-maven-plugin:1.1.2:nondex` under `modules/cli` directory after installation.
   
   **Reason for Flakiness**
   - `CliConfigSubCommandTest#noKey()`:
   In line 38 in the test function, `execute()` is called. In `CliCommandTestBase.java`, `execute()` links to `picocli.CommandLine` in line 53, which in turns calls the function `run()` in `CliConfigSubCommand.java`. The function `run()` then calls `runPipeline()` in `CallExecutionPipeline.java`, which then calls `decorate()` in `ConfigDecorator.java`. Finally, `decorate()` calls a HashMap iterator, which does not guarantee a deterministic order. As a result, line 43 in `CliConfigSubCommandTest#noKey()` can raise an assertion error due to permutation of the string-represented equations.
   - `IgniteCliInterfaceTest$Cluster#initSuccess()`: 
   In line 497 in `IgniteCliInterfaceTest.java`, `cmd.execute()` is called. Subsequently, function `executeUserObject()` (line 1939) of `picocli.CommandLine` is executed, calling `run()` (line 84) of `ClusterCommandSpec.java`. This finally leads to calling `init()` (line 71) in `ClusterApiClient.java`. In the function `init()`, `toJson()` is called (line 84). However, by the Oracle official document, Json-likes are ordered by `getDeclaredFields()`, which does not guarantee any deterministic orders. Therefore, line 514 in `IgniteCliInterfaceTest$Cluster#initSuccess()` can raise an assertion error due to permutation of the byte representations in the strings being compared.
   
   **Sample Failure**
   - `CliConfigSubCommandTest#noKey()`:
   ```
   [ERROR] Failures:
   [ERROR]   CliConfigSubCommandTest.noKey:43
   expected:
     "ignite.cluster-url=test_cluster_url
     ignite.jdbc-url=test_jdbc_url
     "
    but was:
     "ignite.jdbc-url=test_jdbc_url
     ignite.cluster-url=test_cluster_url
     "
   ```
   
   -`IgniteCliInterfaceTest$Cluster#initSuccess()`: 
   ```
   [ERROR] Failures:
   [ERROR]   IgniteCliInterfaceTest$Cluster.initSuccess:514
   Expected: is "{\"metaStorageNodes\":[\"node1ConsistentId\",\"node2ConsistentId\"],\"cmgNodes\":[\"node2ConsistentId\",\"node3ConsistentId\"],\"clusterName\":\"cluster\"}"
        but: was "{\"metaStorageNodes\":[\"node1ConsistentId\",\"node2ConsistentId\"],\"clusterName\":\"cluster\",\"cmgNodes\":[\"node2ConsistentId\",\"node3ConsistentId\"]}"
   ```
   
   ** Proposed Fixes **
   - To deal with different possible iteration orders with respect to `HashMap`, enumerate the 2 possible permutations.
   - To deal with comparisons of JSON-formatted strings, use `JsonParser()` provided by the Google `Gson` library.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [ignite-3] PakhomovAlexander commented on pull request #888: IGNITE-17223 Fix indeterminate unit test results due to order dependencies

[GitBox <gi...@apache.org>]

PakhomovAlexander commented on PR #888:
URL: https://github.com/apache/ignite-3/pull/888#issuecomment-1174851952

   LGTM 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [ignite-3] asfgit closed pull request #888: IGNITE-17223 Fix indeterminate unit test results due to order dependencies

[GitBox <gi...@apache.org>]

asfgit closed pull request #888: IGNITE-17223 Fix indeterminate unit test results due to order dependencies
URL: https://github.com/apache/ignite-3/pull/888


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@ignite.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org