You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by "git-hulk (via GitHub)" <gi...@apache.org> on 2025/09/02 11:57:53 UTC
[I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
git-hulk opened a new issue, #3157:
URL: https://github.com/apache/kvrocks/issues/3157
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.
### Version
unstable
### Minimal reproduce step
Run the test TestWaitCommand, specifically TestWaitCommand/WAIT_with_timeout_should_not_block_indefinitely. Observe the WAIT command behavior during test execution.
### What did you expect to see?
WAIT command should respect the timeout and not block indefinitely.
### What did you see instead?
Test failed with error: WAIT command blocked indefinitely
wait_test.go:110: WAIT command blocked indefinitely
FAIL
exit status 1
### Anything Else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "jihuayu (via GitHub)" <gi...@apache.org>.
jihuayu closed issue #3157: Flaky test in WAIT command blocked indefinitely during test
URL: https://github.com/apache/kvrocks/issues/3157
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "HermanZeng (via GitHub)" <gi...@apache.org>.
HermanZeng commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-5105317266
Yes, that is my conclusion. The production failure mode described in this issue appears to have already been addressed by #3220, while #3157 was simply left open afterward.
I prepared a focused regression test and verified that it distinguishes the behavior before and after #3220. However, reliably constructing the stale-ACK state requires deliberately hitting the ACK rate-limiting window, which makes the test relatively complex and timing-sensitive.
After considering the maintenance cost, I do not think adding this test solely for historical regression coverage is justified. I therefore suggest closing #3157 as resolved by #3220, unless you or other maintainers would still prefer the additional coverage.
Thank you for reviewing the investigation and for clarifying the policy on test-only PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "HermanZeng (via GitHub)" <gi...@apache.org>.
HermanZeng commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-5084787427
Hi, I noticed that PR #3373 was closed and this issue was unassigned before being added to #3516. I'd like to pick this up.
I plan to keep the patch focused: replace the fixed sleeps in `TestWaitBlockExecutingCommand` with a state-based wait for `blocked_clients`, then run the focused test repeatedly. I'll reference and credit #3373 in the PR. May I take this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "HermanZeng (via GitHub)" <gi...@apache.org>.
HermanZeng commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-5096092444
Update after reproducing the historical failure:
My initial plan followed #3373, but that patch targets the later TestWaitBlockExecutingCommand v2/v3 failure. It does not address the original line-110 WAIT timeout reported by this issue, so I will not include that change.
For the original symptom, I checked the issue-era unstable commit dd979ac9 with the same Clang TSan configuration and constructed the relevant state explicitly: the replica has applied the latest write, while the master still records a stale replica ACK and there are no further writes.
Results with the same focused regression test:
- dd979ac9 before #3220: 3/3 failed; WAIT returned 0 after its timeout.
- dd979ac9 plus only #3220: 3/3 passed; the idle replication heartbeat advanced the ACK and WAIT returned 1.
- current unstable: repeated Release and Clang TSan runs pass, and the complete WAIT package passes under Clang TSan.
This supports #3220 as the later production fix for the ACK-starvation failure mode consistent with the original report. #3220 did not add regression coverage, so I now have a focused test-only change that explicitly preserves this behavior.
Would a test-only PR for this historical failure mode, referencing #3157 and #3220, be welcome? Also, should that PR close #3157, or should it only reference the issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
git-hulk commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-3331364629
```
--- FAIL: TestWaitBlockExecutingCommand (3.34s)
wait_test.go:279:
Error Trace: /home/runner/work/kvrocks/kvrocks/tests/gocase/unit/wait/wait_test.go:279
Error: Not equal:
expected: "v3"
actual : "v2"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-v3
+v2
Test: TestWaitBlockExecutingCommand
FAIL
exit status 1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "jihuayu (via GitHub)" <gi...@apache.org>.
jihuayu commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-5100113055
@HermanZeng Thank you for your patience and detailed explanation.
I would like to confirm whether I understood you correctly: do you mean that this issue has already been fixed, and the only missing part is the corresponding test case? It just has not been closed yet.
PRs that only add tests are welcome. The key question is whether we actually need this test case. From my perspective, either adding it or leaving it out is acceptable. You can submit the PR, and if there are no objections from others, we can merge it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
git-hulk commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-3775562621
@ShanukaGallage thank you, assigned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "ShanukaGallage (via GitHub)" <gi...@apache.org>.
ShanukaGallage commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-3773945052
Hi @git-hulk,
I am interested in looking into this flaky test. I noticed the context from the previous PR #3197, and I would like to pick this up.
I will start by analyzing `tests/gocase/unit/wait/wait_test.go` and attempting to reproduce the blocking behavior locally to identify the race condition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "git-hulk (via GitHub)" <gi...@apache.org>.
git-hulk commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-5086570221
@HermanZeng Assigned, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Flaky test in WAIT command blocked indefinitely during test [kvrocks]
Posted by "jihuayu (via GitHub)" <gi...@apache.org>.
jihuayu commented on issue #3157:
URL: https://github.com/apache/kvrocks/issues/3157#issuecomment-5115794432
@HermanZeng Thank you for your suggestion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org