You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by "KangweiZhu (via GitHub)" <gi...@apache.org> on 2025/10/09 04:42:37 UTC
[PR] Fix Flaky Tests in GraphSONTypedCompatibilityTest [tinkerpop]
KangweiZhu opened a new pull request, #3237:
URL: https://github.com/apache/tinkerpop/pull/3237
We observed several tests in GraphSONTypedCompatibilityTest that exhibited flaky behavior when executed with NonDex. Specifically speaking, we can reproduce them by using the following commands.
```
mvn clean install -DskipTests -Drat.skip=true
```
* Test shouldReadWriteEdge[expect(v2)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteEdge[expect(v2)]" -Drat.skip=true
```
* Test shouldReadWriteEdge[expect(v3)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteEdge[expect(v3)]" -Drat.skip=true
```
* Test shouldReadWritePath[expect(v2)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWritePath[expect(v2)]" -Drat.skip=true
```
* Test shouldReadWritePath[expect(v3)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWritePath[expect(v3)]" -Drat.skip=true
```
* Test shouldReadWriteProperty[expect(v2)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteProperty[expect(v2)]" -Drat.skip=true
```
* Test shouldReadWriteProperty[expect(v3)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteProperty[expect(v3)]" -Drat.skip=true
```
* Test shouldReadWriteTraverser[expect(v2)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteTraverser[expect(v2)]" -Drat.skip=true
```
* Test shouldReadWriteTraverser[expect(v3)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteTraverser[expect(v3)]" -Drat.skip=true
```
* Test shouldReadWriteVertexProperty[expect(v2)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteVertexProperty[expect(v2)]" -Drat.skip=true
```
* Test shouldReadWriteVertexProperty[expect(v3)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteVertexProperty[expect(v3)]" -Drat.skip=true
```
* Test shouldReadWriteVertex[expect(v2)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteVertex[expect(v2)]" -Drat.skip=true
```
* Test shouldReadWriteVertex[expect(v3)]
```bash
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONTypedCompatibilityTest#shouldReadWriteVertex[expect(v3)]" -Drat.skip=true
```
And the error should be something like this:
```
[ERROR] Failures:
[ERROR] GraphSONTypedCompatibilityTest>AbstractTypedCompatibilityTest.shouldReadWriteEdge:322 expected:<e[17][7-develops->10]> but was:<e[13][1-develops->10]>
[ERROR] GraphSONTypedCompatibilityTest>AbstractTypedCompatibilityTest.shouldReadWriteEdge:322 expected:<e[17][7-develops->10]> but was:<e[13][1-develops->10]>
```
Upon our investigation, the root cause is the use of:
```
protected Map<Object, Vertex> vertices = new ConcurrentHashMap<>();
protected Map<Object, Edge> edges = new ConcurrentHashMap<>();
```
in TinkerGraph.java, which does not guarantee a deterministic order.
The simplest fix would be to replace ConcurrentHashMap with LinkedHashMap, as we did in a previous PR. We've confirmed that this change could remove the flakiness of these tests. However, we are concerned that such a change might introduce unintended side effects in the code under test. Another possible fix would be to deterministically select a fixed id, but that approach would make the test become sensitive to future implementation changes. Thus, we decided to use this sorting-based approach to deflake this test.
<!--
Thanks for contributing! Reminders:
+ TARGET the earliest branch where you want the change
3.7-dev -> 3.7.5 (non-breaking only)
3.8-dev -> 3.8.0
master -> 4.0.0
+ Committers will MERGE the PR forward to newer versions
+ ADD entry to the CHANGELOG.asciidoc for the targeted version
Do not reference a JIRA number there
+ ADD JIRA number to title and link in description
+ PRs requires 3 +1s from committers OR
1 +1 and 7 day wait to merge.
+ MORE details: https://s.apache.org/rtnal
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Deflake tests in GraphSONTypedCompatibilityTest, GraphSONUntypedCompatibilityTest and GraphBinaryCompatibilityTest [tinkerpop]
Posted by "Cole-Greer (via GitHub)" <gi...@apache.org>.
Cole-Greer merged PR #3237:
URL: https://github.com/apache/tinkerpop/pull/3237
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Deflake tests in GraphSONTypedCompatibilityTest, GraphSONUntypedCompatibilityTest and GraphBinaryCompatibilityTest [tinkerpop]
Posted by "KangweiZhu (via GitHub)" <gi...@apache.org>.
KangweiZhu commented on PR #3237:
URL: https://github.com/apache/tinkerpop/pull/3237#issuecomment-3439758996
We made another change that deflakes serveral tests in GraphSONUntypedCompatibilityTest.
In addition, we found the scope of this fix is broader than we initially thought, so we updated the title of this PR and modified the commit message.
This pr actually also fixed all flaky tests in GraphSONUntypedCompatibilityTest and GraphBinaryCompatibilityTest, and could be verified by:
```
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUntypedCompatibilityTest" -Drat.skip=true
```
```
mvn clean -pl gremlin-util edu.illinois:nondex-maven-plugin:2.2.1:nondex -Dtest="org.apache.tinkerpop.gremlin.structure.io.graphbinary.GraphBinaryCompatibilityTest" -Drat.skip=true
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Flaky Tests in GraphSONTypedCompatibilityTest [tinkerpop]
Posted by "andreachild (via GitHub)" <gi...@apache.org>.
andreachild commented on code in PR #3237:
URL: https://github.com/apache/tinkerpop/pull/3237#discussion_r2421237749
##########
gremlin-util/src/test/java/org/apache/tinkerpop/gremlin/structure/io/Model.java:
##########
@@ -115,12 +116,22 @@ private Model() {
addCoreEntry(new java.sql.Timestamp(1481750076295L), "Timestamp", "");
addCoreEntry(UUID.fromString("41d2e28a-20a4-4ab0-b379-d810dede3786"), "UUID");
- addGraphStructureEntry(graph.edges().next(), "Edge", "");
- addGraphStructureEntry(g.V().out().out().path().next(), "Path", "");
- addGraphStructureEntry(graph.edges().next().properties().next(), "Property", "");
+ addGraphStructureEntry(IteratorUtils.list(graph.edges()).stream()
+ .sorted((e1, e2) -> Integer.compare((Integer)e1.id(), (Integer)e2.id()))
+ .iterator().next(), "Edge", "");
+ addGraphStructureEntry(g.V().order().by(T.id).out().out().path().next(), "Path", "");
+ addGraphStructureEntry(IteratorUtils.list(graph.edges()).stream()
+ .sorted((e1, e2) -> Integer.compare((Integer)e1.id(), (Integer)e2.id()))
+ .iterator().next().properties().next(), "Property", "");
Review Comment:
Should the properties be sorted as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Flaky Tests in GraphSONTypedCompatibilityTest [tinkerpop]
Posted by "Cole-Greer (via GitHub)" <gi...@apache.org>.
Cole-Greer commented on PR #3237:
URL: https://github.com/apache/tinkerpop/pull/3237#issuecomment-3407420484
VOTE +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix Flaky Tests in GraphSONTypedCompatibilityTest [tinkerpop]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #3237:
URL: https://github.com/apache/tinkerpop/pull/3237#issuecomment-3386973288
## [Codecov](https://app.codecov.io/gh/apache/tinkerpop/pull/3237?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 74.24%. Comparing base ([`9b46b67`](https://app.codecov.io/gh/apache/tinkerpop/commit/9b46b6777d2fa250e41daacf2fa4554605aff53a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`5364eee`](https://app.codecov.io/gh/apache/tinkerpop/commit/5364eeebeb74199c42a69fc972c523f9bb59d1fc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 382 commits behind head on 3.7-dev.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## 3.7-dev #3237 +/- ##
=============================================
- Coverage 76.14% 74.24% -1.91%
=============================================
Files 1084 28 -1056
Lines 65160 5917 -59243
Branches 7285 0 -7285
=============================================
- Hits 49616 4393 -45223
+ Misses 12839 1322 -11517
+ Partials 2705 202 -2503
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/tinkerpop/pull/3237?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org