You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hugegraph.apache.org by GitBox <gi...@apache.org> on 2022/12/10 22:48:01 UTC
[GitHub] [incubator-hugegraph-toolchain] aditya-kumbhar opened a new pull request, #398: Fix flaky tests in IndexLabelTest due to non deterministic JSON key ordering
aditya-kumbhar opened a new pull request, #398:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/398
### Motivation
IndexLabelTest has the following tests that are flaky, detected using the [NonDex](https://github.com/TestingResearchIllinois/NonDex) tool:
org.apache.hugegraph.unit.IndexLabelTest#testIndexLabel
org.apache.hugegraph.unit.IndexLabelTest#testIndexLabelV49
org.apache.hugegraph.unit.IndexLabelTest#testIndexLabelV56
The tests fail due to non-deterministic ordering of the properties in the JSON string created by the `JsonUtil.toJson()` method.
### Command to reproduce
mvn -pl hugegraph-client edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.hugegraph.unit.IndexLabelTest
### Log
```
[ERROR] IndexLabelTest.testIndexLabel:47 expected:<...ame":"personByAge","[id":0,"check_exist":true,"user_data":{},"base_type":"VERTEX_LABEL","base_value":"person","index_type":"SECONDARY","fields":["age"],"rebuild":true]}> but was:<...ame":"personByAge","[check_exist":true,"user_data":{},"id":0,"fields":["age"],"base_value":"person","rebuild":true,"index_type":"SECONDARY","base_type":"VERTEX_LABEL"]}>
[ERROR] IndexLabelTest.testIndexLabelV49:66 expected:<{"id":0,"[name":"personByAge","check_exist":true],"base_type":"VERTEX...> but was:<{"id":0,"[check_exist":true,"name":"personByAge"],"base_type":"VERTEX...>
[ERROR] IndexLabelTest.testIndexLabelV56:88 expected:<{"[id":0,"name":"personByAge","check_exist":true,"user_data":{},"base_type":"VERTEX_LABEL","base_value":"person","index_type":"SECONDARY]","fields":["age"]}> but was:<{"[name":"personByAge","id":0,"check_exist":true,"user_data":{},"index_type":"SECONDARY","base_type":"VERTEX_LABEL","base_value":"person]","fields":["age"]}>
```
### Changes
Using the jackson ObjectMapper, comparing the JSONNodes instead of raw json strings would make the test deterministic, as the order of the properties will not be compared.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph-toolchain] aditya-kumbhar commented on pull request #398: Fix flaky tests in IndexLabelTest due to non deterministic JSON key ordering
Posted by GitBox <gi...@apache.org>.
aditya-kumbhar commented on PR #398:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/398#issuecomment-1345458817
A flaky test would have no direct impact on production environment, but the tests themselves may fail if moved/run in different java environments due to assumptions made about the determinism of the Java APIs. In this case the order of the elements of the json returned by `JsonUtil.toJson()` is assumed to be same as the predefined order defined in the expected string `json`.
The [Nondex](https://github.com/TestingResearchIllinois/NonDex) tool helps in identifying such assumptions by exploring different behaviors of non-deterministic Java APIs and reports the flaky tests.
Fixing the flaky tests would make the behavior of the tests consistent across java environments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph-toolchain] imbajin commented on pull request #398: Fix flaky tests in IndexLabelTest due to non deterministic JSON key ordering
Posted by GitBox <gi...@apache.org>.
imbajin commented on PR #398:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/398#issuecomment-1345454258
thanks,we will take a look for it soon
and what's the influence in production env?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph-toolchain] javeme merged pull request #398: Fix flaky tests in IndexLabelTest due to non deterministic JSON key ordering
Posted by GitBox <gi...@apache.org>.
javeme merged PR #398:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/398
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-hugegraph-toolchain] codecov[bot] commented on pull request #398: Fix flaky tests in IndexLabelTest due to non deterministic JSON key ordering
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #398:
URL: https://github.com/apache/incubator-hugegraph-toolchain/pull/398#issuecomment-1345454779
# [Codecov](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#398](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2e88ff7) into [master](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/commit/1c47673eb9511d3eff0d0901012a69d7a0148cdc?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1c47673) will **increase** coverage by `0.03%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #398 +/- ##
============================================
+ Coverage 62.55% 62.58% +0.03%
+ Complexity 1866 894 -972
============================================
Files 260 91 -169
Lines 9405 4383 -5022
Branches 872 515 -357
============================================
- Hits 5883 2743 -3140
+ Misses 3140 1437 -1703
+ Partials 382 203 -179
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...va/org/apache/hugegraph/structure/auth/Access.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3N0cnVjdHVyZS9hdXRoL0FjY2Vzcy5qYXZh) | | |
| [...org/apache/hugegraph/api/schema/IndexLabelAPI.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS9zY2hlbWEvSW5kZXhMYWJlbEFQSS5qYXZh) | | |
| [...hugegraph/exception/InvalidOperationException.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2V4Y2VwdGlvbi9JbnZhbGlkT3BlcmF0aW9uRXhjZXB0aW9uLmphdmE=) | | |
| [...org/apache/hugegraph/structure/traverser/Kout.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3N0cnVjdHVyZS90cmF2ZXJzZXIvS291dC5qYXZh) | | |
| [.../apache/hugegraph/structure/auth/HugeResource.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3N0cnVjdHVyZS9hdXRoL0h1Z2VSZXNvdXJjZS5qYXZh) | | |
| [...ugegraph/structure/traverser/KneighborRequest.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3N0cnVjdHVyZS90cmF2ZXJzZXIvS25laWdoYm9yUmVxdWVzdC5qYXZh) | | |
| [...hugegraph/serializer/direct/RocksDBSerializer.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3NlcmlhbGl6ZXIvZGlyZWN0L1JvY2tzREJTZXJpYWxpemVyLmphdmE=) | | |
| [...ava/org/apache/hugegraph/structure/auth/Group.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3N0cnVjdHVyZS9hdXRoL0dyb3VwLmphdmE=) | | |
| [...g/apache/hugegraph/api/traverser/KneighborAPI.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL2FwaS90cmF2ZXJzZXIvS25laWdoYm9yQVBJLmphdmE=) | | |
| [...g/apache/hugegraph/structure/auth/LoginResult.java](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-aHVnZWdyYXBoLWNsaWVudC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvaHVnZWdyYXBoL3N0cnVjdHVyZS9hdXRoL0xvZ2luUmVzdWx0LmphdmE=) | | |
| ... and [159 more](https://codecov.io/gh/apache/incubator-hugegraph-toolchain/pull/398/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hugegraph.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org