You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by "tju-yxq (via GitHub)" <gi...@apache.org> on 2026/08/28 07:39:42 UTC
[I] Notification outbox can redeliver a completed delivery [rocketmq-dashboard]
tju-yxq opened a new issue, #2661:
URL: https://github.com/apache/rocketmq-dashboard/issues/2661
`NotificationOutboxService` can turn a notification that was already sent into another delivery attempt.
The dispatcher marks a row as `SENDING` and then performs the SMTP or webhook call synchronously. The claim is not renewed while that call is in flight, so a slow SMTP transaction (or a deployment with multiple Studio replicas) can make the row look stale after the one-minute claim window. A second dispatcher can then claim the same row and send the alert again.
There is a second state-machine gap in the same path: after the external call succeeds, the row is marked `DELIVERED` and the delivery audit record is written. If that audit insert fails, the surrounding `try` block treats the bookkeeping exception as a send failure and moves the row back to `RETRY_WAIT`. The next dispatcher retries a notification that has already reached its recipient.
Expected behavior:
- renew a claimed row while its delivery attempt is still running, with the renewal conditional on the current claim token;
- stop changing the row when ownership is lost, rather than allowing a stale worker to overwrite a newer attempt;
- never convert a successfully completed external send into a retry only because audit recording failed;
- record delivery and retry timestamps from the actual completion/failure time.
Please add regression coverage for a slow/in-flight claim being renewed and for an audit write failure after a successful send. The tests should keep the existing at-least-once retry behavior for genuine webhook/SMTP failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Notification outbox can redeliver a completed delivery [rocketmq-dashboard]
Posted by "tju-yxq (via GitHub)" <gi...@apache.org>.
tju-yxq commented on issue #2661:
URL: https://github.com/apache/rocketmq-dashboard/issues/2661#issuecomment-5450060375
Implemented in PR #2670 (`f6e87f5c`). The fix keeps an outbox claim alive during a slow external delivery, prevents a worker that lost the claim from writing a newer worker's state, and treats audit persistence as separate from delivery retry decisions. Regression tests cover renewal, lease loss, and both post-send failure paths. Local backend verification passed: 16 focused tests and the full 1,792-test server suite, with no Checkstyle violations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Notification outbox can redeliver a completed delivery [rocketmq-dashboard]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins commented on issue #2661:
URL: https://github.com/apache/rocketmq-dashboard/issues/2661#issuecomment-5509305914
Resolved by #2670 (merged into rocketmq-studio as f274d6a0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Notification outbox can redeliver a completed delivery [rocketmq-dashboard]
Posted by "lizhimins (via GitHub)" <gi...@apache.org>.
lizhimins closed issue #2661: Notification outbox can redeliver a completed delivery
URL: https://github.com/apache/rocketmq-dashboard/issues/2661
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Notification outbox can redeliver a completed delivery [rocketmq-dashboard]
Posted by "RockteMQ-AI (via GitHub)" <gi...@apache.org>.
RockteMQ-AI commented on issue #2661:
URL: https://github.com/apache/rocketmq-dashboard/issues/2661#issuecomment-5454242874
**Issue Evaluation**
Category: | Status: **Confirmed**
The reported issue has been verified against the current codebase on the branch.
**Root Cause (two state-machine gaps confirmed):**
1. **No claim renewal during in-flight delivery:** calls which sets , then performs the SMTP/webhook call synchronously. The claim is never renewed during this call. If the external call exceeds (1 minute), in another replica will treat the row as stale and re-claim it, causing a duplicate send.
2. **Post-send bookkeeping failure triggers retry of already-delivered notification:** In , after the SMTP/webhook call succeeds (lines 267-269), the code updates status to (line 271-274) and then calls (line 277). If throws, the catch block at line 278 calls , which moves the row to — even though the notification was already delivered to the recipient. The next dispatcher will re-send it.
**Impact:** Duplicate alert notifications via email/webhook, potentially causing alert fatigue or incorrect operational responses.
**Severity:** Medium-High — affects notification reliability and can cause duplicate alerts in production.
**Code references verified:**
- — with claim but no renewal
- — with post-delivery exception routing to
- — moves row to
An automated fix proposal can be generated. Reply to proceed with PR generation.
---
*Automated evaluation by @RockteMQ-AI*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org