You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@seata.apache.org by "XiaoyangCai360 (via GitHub)" <gi...@apache.org> on 2025/12/14 03:53:21 UTC
[PR] test: Fix flaky testIsRaftMode and testToString in MetadataTest [incubator-seata]
XiaoyangCai360 opened a new pull request, #7859:
URL: https://github.com/apache/incubator-seata/pull/7859
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- Please make sure you have read and understood the contributing guidelines -->
- [x] I have read the [CONTRIBUTING.md](https://github.com/apache/incubator-seata/blob/2.x/CONTRIBUTING.md) guidelines.
- [x] I have registered the PR [changes](https://github.com/apache/incubator-seata/tree/2.x/changes).
### Ⅰ. Describe what this PR did
This PR resolves test flakiness in `org.apache.seata.common.metadata.MetadataTest`, specifically:
- `MetadataTest.testToString`
- `MetadataTest.testIsRaftMode`
The flakiness was caused by:
1) the test class sharing a single static `Metadata` instance across tests, making assertions dependent on test execution order (e.g., `storeMode` mutated by other tests); and
2) `testToString` relying on an exact `toString()` output, which is brittle because field/map ordering may vary across executions and under NonDex.
The fix makes the tests deterministic by setting up required state explicitly and using order-independent assertions (instead of strict string equality).
**This PR is test-only and does not modify production code.**
### Ⅱ. Does this pull request fix one issue?
<!-- If that, add "fixes #xxx" below in the next line, for example, fixes #97. -->
### Ⅲ. Why don't you add test cases (unit test/integration test)?
This PR focuses on stabilizing existing unit tests by removing order-dependent assumptions and brittle `toString()` comparisons.
### Ⅳ. Describe how to verify it
Run the affected tests repeatedly under a non-deterministic execution environment such as [NonDex](https://github.com/TestingResearchIllinois/NonDex) to confirm stability.
Example command:
```bash
mvn -pl common \
-am edu.illinois:nondex-maven-plugin:2.1.7:nondex \
-Dtest=org.apache.seata.common.metadata.MetadataTest \
-DnondexRuns=20 \
-DfailIfNoTests=false
```
### Ⅴ. Special notes for reviews
- Test-only change in the `common` module.
- No production code modified.
- Improves CI stability under non-deterministic execution orders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [PR] test: Fix flaky testIsRaftMode and testToString in MetadataTest [incubator-seata]
Posted by "YongGoose (via GitHub)" <gi...@apache.org>.
YongGoose commented on PR #7859:
URL: https://github.com/apache/incubator-seata/pull/7859#issuecomment-3660646868
Please resolve the conflicts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [PR] test: fix flaky testIsRaftMode and testToString in MetadataTest [incubator-seata]
Posted by "xingfudeshi (via GitHub)" <gi...@apache.org>.
xingfudeshi merged PR #7859:
URL: https://github.com/apache/incubator-seata/pull/7859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [PR] test: Fix flaky testIsRaftMode and testToString in MetadataTest [incubator-seata]
Posted by "YongGoose (via GitHub)" <gi...@apache.org>.
YongGoose commented on PR #7859:
URL: https://github.com/apache/incubator-seata/pull/7859#issuecomment-3660645681
> Thanks for the questions and the suggestion!
>
> 1. There is no public CI failure record for these tests. The flakiness was identified using NonDex, which exposes non-deterministic behaviors such as shared static state and order-dependent assertions. Although this issue may not consistently surface in CI today, it represents a latent source of non-deterministic failures that can cause flaky CI behavior under different JVM implementations, execution orders, or future refactoring.
> 2. I agree that using BeforeEach would also address the shared-state issue. However, in this PR, I took a minimal approach to make the affected tests self-contained by explicitly setting the required state (storeMode) and avoiding brittle, order-dependent toString() equality checks.
Ok. I understand
Nice catch :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [PR] test: Fix flaky testIsRaftMode and testToString in MetadataTest [incubator-seata]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #7859:
URL: https://github.com/apache/incubator-seata/pull/7859#issuecomment-3650194006
## [Codecov](https://app.codecov.io/gh/apache/incubator-seata/pull/7859?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 70.93%. Comparing base ([`1b2a486`](https://app.codecov.io/gh/apache/incubator-seata/commit/1b2a4860b5404e01e599486b400e8a4eadaf02a1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`94dc794`](https://app.codecov.io/gh/apache/incubator-seata/commit/94dc7944f67419c652a124c2e7d18217e38ac352?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## 2.x #7859 +/- ##
============================================
+ Coverage 70.90% 70.93% +0.02%
Complexity 992 992
============================================
Files 1322 1322
Lines 50322 50322
Branches 5946 5946
============================================
+ Hits 35683 35695 +12
+ Misses 11693 11680 -13
- Partials 2946 2947 +1
```
[see 12 files with indirect coverage changes](https://app.codecov.io/gh/apache/incubator-seata/pull/7859/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
[](https://app.codecov.io/gh/apache/incubator-seata/pull/7859?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org
Re: [PR] test: Fix flaky testIsRaftMode and testToString in MetadataTest [incubator-seata]
Posted by "XiaoyangCai360 (via GitHub)" <gi...@apache.org>.
XiaoyangCai360 commented on PR #7859:
URL: https://github.com/apache/incubator-seata/pull/7859#issuecomment-3658750971
Thanks for the questions and the suggestion!
1. There is no public CI failure record for these tests. The flakiness was identified using NonDex, which exposes non-deterministic behaviors such as shared static state and order-dependent assertions. Although this issue may not consistently surface in CI today, it represents a latent source of non-deterministic failures that can cause flaky CI behavior under different JVM implementations, execution orders, or future refactoring.
2. I agree that using @BeforeEach would also address the shared-state issue. However, in this PR, I took a minimal approach to make the affected tests self-contained by explicitly setting the required state (storeMode) and avoiding brittle, order-dependent toString() equality checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@seata.apache.org
For additional commands, e-mail: notifications-help@seata.apache.org