You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "smengcl (via GitHub)" <gi...@apache.org> on 2026/08/13 07:57:38 UTC
[PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
smengcl opened a new pull request, #11014:
URL: https://github.com/apache/ozone/pull/11014
Generated-by: Claude Code (Opus 4.8)
## What changes were proposed in this pull request?
`TestSnapshotDiffManager#testLoadJobsOnStartUp` fails intermittently with `expected: <DONE> but was: <IN_PROGRESS>`.
The test stubs `generateSnapshotDiffReport` with a `doAnswer` that writes `DONE` to the DB inside the answer body, calls `loadJobsOnStartUp()`, then waits with `attempt(() -> verify(spy, atLeast(1)).generateSnapshotDiffReport(...))` before reading the job back and asserting `DONE`.
Mockito records an invocation before it runs the stubbed answer, so the `verify(...)` wait can pass as soon as the worker thread enters `generateSnapshotDiffReport`, while the answer is still running and has not yet persisted `DONE`. The main thread then reads the job and observes the original `IN_PROGRESS`, so the assertion fails.
The fix waits on the actual observable end state the test asserts: it polls `getSnapshotDiffJobFromDb(...).getStatus()` until it is `DONE`, instead of waiting on the Mockito invocation. This is deterministic by construction because the read the assertion depends on is exactly what is waited on. No production code changes.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16170
## How was this patch tested?
Ran the affected test locally against the change:
```
mvn -pl :ozone-manager -am test -Dtest='TestSnapshotDiffManager#testLoadJobsOnStartUp' -DskipShade -DskipRecon -DskipDocs -Dsurefire.failIfNoSpecifiedTests=false
```
A single green run cannot prove non-flakiness, so the fix is made deterministic by construction: the wait polls the same persisted job status that the subsequent assertion reads, closing the window between Mockito recording the invocation and the stubbed answer writing `DONE` to the DB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #11014:
URL: https://github.com/apache/ozone/pull/11014#issuecomment-5772331238
Thanks @jojochuang @sadanand48 for the reviews.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #11014:
URL: https://github.com/apache/ozone/pull/11014#issuecomment-5403227960
Merged. Thanks @smengcl @sadanand48
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #11014:
URL: https://github.com/apache/ozone/pull/11014
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "sadanand48 (via GitHub)" <gi...@apache.org>.
sadanand48 commented on PR #11014:
URL: https://github.com/apache/ozone/pull/11014#issuecomment-5313654926
Thanks @smengcl for the patch, could you please run [flaky-test-check](https://github.com/smengcl/hadoop-ozone/actions/workflows/intermittent-test-check.yml) for this test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #11014:
URL: https://github.com/apache/ozone/pull/11014#issuecomment-5361474647
> Thanks @smengcl for the patch, could you please run [flaky-test-check](https://github.com/smengcl/hadoop-ozone/actions/workflows/intermittent-test-check.yml) for this test
Thanks @sadanand48 for taking a look.
10x10 ran and passed: https://github.com/smengcl/hadoop-ozone/actions/runs/32387971098/job/96487939435#step:7:23
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #11014:
URL: https://github.com/apache/ozone/pull/11014#issuecomment-5363689810
@sadanand48 Thanks for reviewing. I kicked off flaky-test-check for `TestSnapshotDiffManager#testLoadJobsOnStartUp` on my fork (10 splits x 10 iterations, 100 runs total, java25): https://github.com/smengcl/hadoop-ozone/actions/runs/32432617806
I will report the result here once it finishes.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-16170. Intermittent failure in TestSnapshotDiffManager#testLoadJobsOnStartUp [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #11014:
URL: https://github.com/apache/ozone/pull/11014#issuecomment-5363903634
flaky-test-check passed: all 10 splits green, 0 failures across 100 runs (10 splits x 10 iterations, java25).
Run: https://github.com/smengcl/hadoop-ozone/actions/runs/32432617806
This matches the intent of the fix: the wait now polls the persisted job status until `DONE` instead of waiting on the Mockito invocation, so it no longer returns before the stubbed `doAnswer` writes `DONE` to the DB.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org