You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "Anshul-creator (via GitHub)" <gi...@apache.org> on 2025/10/28 04:56:29 UTC

[PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension (OD-Vic flakiness due to duplicate DemoImpl registration) [dubbo]

Anshul-creator opened a new pull request, #15744:
URL: https://github.com/apache/dubbo/pull/15744

   ## What is the purpose of the change?
   
   This PR fixes **order-dependent flakiness (OD-Vic)** in `ExtensionLoaderTest.testInjectExtension`
   
   This test intermittently fails under randomized test execution orders due to multiple DemoImpl instances being registered in the shared ScopeBeanFactory of ApplicationModel.
   
   ## Root Cause
   
   `testInjectExtension` and related tests (`testGetOrDefaultExtension`) both interacted with the same shared `ScopeBeanFactory`.
   Because bean registration was not idempotent, repeated or parallel test execution sometimes resulted in duplicate `DemoImpl` entries, leading to inconsistent injection results or assertion failures.
   
   ## What's Changed
   - Added defensive logic to check existing DemoImpl beans before registering a new one.
   - Made registration idempotent by registering only one instance (demo-test-singleton) if none exists.
   - Ensured subsequent tests reuse the same canonical bean reference.
   - Updated testGetOrDefaultExtension to follow the same consistent setup.
   
   These changes eliminate shared-state interference and make the test deterministic under any order or random seed.
   
   ## Verification
   
   You can try running the loop below on pre-fix and post-fix code:
   ```bash
   fail=0; total=0
   for s in 7138 7351 7892 11170 15857 15869 17721 20421 21699 22631 24059 29905; do
     echo "SEED=$s"
     total=$((total+1))
     SEED=$s ./mvnw -pl dubbo-common -DfailIfNoTests=true -DforkCount=1 -DreuseForks=false \
       -Dsurefire.runOrder=random \
       -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random \
       -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random \
       -Djunit.jupiter.testclass.order.random.seed="$s" \
       -Djunit.jupiter.testmethod.order.random.seed="$s" \
       -Dtest=org.apache.dubbo.common.extension.ExtensionLoaderTest test \
       >/dev/null || { echo "** FAILED on SEED=$s **"; fail=$((fail+1)); }
   done
   echo
   echo "===== SUMMARY ====="
   echo "Total seeds: $total"
   echo "Failed:      $fail"
   echo "Passed:      $((total - fail))"
   echo "==================="
   ```
   
   The seeds were found by using a script I developed locally. Sometimes not all seeds fail on execution, but some will always fail on pre-fix code. However, on post-fix code none of the seeds fail.
   
   ## Checklist
   - [x] Make sure there is a [GitHub_issue](https://github.com/apache/dubbo/issues) field for the change.
   - [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
   - [x] Write necessary unit-test to verify your logic correction. If the new feature or significant change is committed, please remember to add sample in [dubbo samples](https://github.com/apache/dubbo-samples) project.
   - [x] Make sure gitHub actions can pass. [Why the workflow is failing and how to fix it?](../CONTRIBUTING.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org


Re: [PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension [dubbo]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #15744:
URL: https://github.com/apache/dubbo/pull/15744#issuecomment-3454631406

   ## [Codecov](https://app.codecov.io/gh/apache/dubbo/pull/15744?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 32.10%. Comparing base ([`aed93d5`](https://app.codecov.io/gh/apache/dubbo/commit/aed93d5abe21bc7d30b7de08929205a4dbb3c16a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`2ba50b1`](https://app.codecov.io/gh/apache/dubbo/commit/2ba50b127efc9ef6eee98ccb8a6f939a115d9826?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   > :exclamation:  There is a different number of reports uploaded between BASE (aed93d5) and HEAD (2ba50b1). Click for more details.
   > 
   > <details><summary>HEAD has 6 uploads less than BASE</summary>
   >
   >| Flag | BASE (aed93d5) | HEAD (2ba50b1) |
   >|------|------|------|
   >|unit-tests-java8|1|0|
   >|unit-tests-java11|1|0|
   >|unit-tests-java21|1|0|
   >|unit-tests-java17|1|0|
   >|integration-tests-java21|1|0|
   >|integration-tests-java8|1|0|
   ></details>
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@              Coverage Diff              @@
   ##                3.3   #15744       +/-   ##
   =============================================
   - Coverage     60.75%   32.10%   -28.65%     
   - Complexity       15    10128    +10113     
   =============================================
     Files          1938     1937        -1     
     Lines         88627    88608       -19     
     Branches      13381    13296       -85     
   =============================================
   - Hits          53842    28445    -25397     
   - Misses        29255    55999    +26744     
   + Partials       5530     4164     -1366     
   ```
   
   | [Flag](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [integration-tests-java21](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [integration-tests-java8](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [samples-tests-java21](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `32.04% <ø> (-0.02%)` | :arrow_down: |
   | [samples-tests-java8](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `29.82% <ø> (+0.01%)` | :arrow_up: |
   | [unit-tests-java11](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unit-tests-java17](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unit-tests-java21](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   | [unit-tests-java8](https://app.codecov.io/gh/apache/dubbo/pull/15744/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/dubbo/pull/15744?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org


Re: [PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension [dubbo]

Posted by "Anshul-creator (via GitHub)" <gi...@apache.org>.
Anshul-creator commented on PR #15744:
URL: https://github.com/apache/dubbo/pull/15744#issuecomment-3492632768

   Thanks for the review!
   
   I believe there might be a small misunderstanding...this PR doesn’t introduce new behavior that needs its own unit test. It fixes order-dependent flakiness in the existing test `ExtensionLoaderTest.testInjectExtension`
   
   Before this change, the `ExtensionLoader` classes' tests, particularly `testGetOrDefaultExtension` (polluter) and `testInjectExtension` (victim) interfered with each other via shared `ScopeBeanFactory` state. Running the shell command below exposes this flakiness:
   
   ```bash
   fail=0; total=0
   for s in 7138 7351 7892 11170 15857 15869 17721 20421 21699 22631 24059 29905; do
     echo "SEED=$s"
     total=$((total+1))
     SEED=$s ./mvnw -pl dubbo-common -DfailIfNoTests=true -DforkCount=1 -DreuseForks=false \
       -Dsurefire.runOrder=random \
       -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random \
       -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random \
       -Djunit.jupiter.testclass.order.random.seed="$s" \
       -Djunit.jupiter.testmethod.order.random.seed="$s" \
       -Dtest=org.apache.dubbo.common.extension.ExtensionLoaderTest test \
       >/dev/null || { echo "** FAILED on SEED=$s **"; fail=$((fail+1)); }
   done
   echo
   echo "===== SUMMARY ====="
   echo "Total seeds: $total"
   echo "Failed:      $fail"
   echo "Passed:      $((total - fail))"
   echo "==================="
   ```
   
   These seeds reliably reproduce the flakiness on the pre-fix code.
   
   I attempted to simplify this to a single command (e.g., running the polluter → victim pair deterministically), but Surefire didn’t consistently preserve the inter-test interference needed to trigger the bug. The above loop was therefore the most reliable way to demonstrate the issue.
   
   After this fix, none of these seeds fail. The test is now stable under any random execution order.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org


Re: [PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension [dubbo]

Posted by "zrlw (via GitHub)" <gi...@apache.org>.
zrlw commented on PR #15744:
URL: https://github.com/apache/dubbo/pull/15744#issuecomment-3483549516

   ```
   fail=0; total=0
   for s in 7138 7351 7892 11170 15857 15869 17721 20421 21699 22631 24059 29905; do
     echo "SEED=$s"
     total=$((total+1))
     SEED=$s ./mvnw -pl dubbo-common -DfailIfNoTests=true -DforkCount=1 -DreuseForks=false \
       -Dsurefire.runOrder=random \
       -Djunit.jupiter.testclass.order.default=org.junit.jupiter.api.ClassOrderer\$Random \
       -Djunit.jupiter.testmethod.order.default=org.junit.jupiter.api.MethodOrderer\$Random \
       -Djunit.jupiter.testclass.order.random.seed="$s" \
       -Djunit.jupiter.testmethod.order.random.seed="$s" \
       -Dtest=org.apache.dubbo.common.extension.ExtensionLoaderTest test \
       >/dev/null || { echo "** FAILED on SEED=$s **"; fail=$((fail+1)); }
   done
   echo
   echo "===== SUMMARY ====="
   echo "Total seeds: $total"
   echo "Failed:      $fail"
   echo "Passed:      $((total - fail))"
   echo "==================="
   ```
   
   It might be better that adding an unit test to prove this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org


Re: [PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension [dubbo]

Posted by "Anshul-creator (via GitHub)" <gi...@apache.org>.
Anshul-creator commented on PR #15744:
URL: https://github.com/apache/dubbo/pull/15744#issuecomment-3498568661

   I will look into this and add a comment when I made changes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org


Re: [PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension [dubbo]

Posted by "zrlw (via GitHub)" <gi...@apache.org>.
zrlw commented on PR #15744:
URL: https://github.com/apache/dubbo/pull/15744#issuecomment-3668019239

   > I will look into this and add a comment when I made changes
   
   Any progress?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org


Re: [PR] Fix: Stabilize ExtensionLoaderTest.testInjectExtension [dubbo]

Posted by "zrlw (via GitHub)" <gi...@apache.org>.
zrlw commented on PR #15744:
URL: https://github.com/apache/dubbo/pull/15744#issuecomment-3494944248

   i mean you could write an junit test by calling  java ```Process``` to run your script and assert its result.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org