[I] [Bug] Proxy removes transaction subscriptions before the first producer heartbeat [rocketmq]

["qianye1001 (via GitHub)" <gi...@apache.org>]

qianye1001 opened a new issue, #11170:
URL: https://github.com/apache/rocketmq/issues/11170

   ### Runtime platform environment
   
   Proxy cluster mode with a Remoting transaction producer. The state transition can be reproduced deterministically in a service-level unit test, without a running broker or timing sleeps.
   
   ### RocketMQ version
   
   Current `develop`: `80e1ae55773c2330d2005b86f020ed028f649e94`.
   
   ### JDK Version
   
   Local regression validation: Amazon Corretto 11.0.23, Maven 3.9.8, macOS arm64.
   
   ### Describe the Bug
   
   `ClusterTransactionService.scanProducerHeartBeat()` deletes a transaction subscription as soon as `ProducerManager.groupOnline(group)` returns false. A transaction send can create this subscription before the producer's first heartbeat registers its channel in the proxy.
   
   If the scan runs in that interval, returning `null` from `groupClusterData.computeIfPresent()` removes the group-to-cluster mapping. A later producer heartbeat only updates `ProducerManager`; it does not rebuild the deleted transaction subscription. If the producer sends no further transaction messages, subsequent scans cannot register that group with the broker, so transaction checks cannot reach the producer.
   
   This is a follow-up to #8316, which reports the same race and was closed by the stale bot on 2026-09-15. It remains reproducible in the current implementation. See also #5862 and its fix #5865: preventing indefinite heartbeats for producers that exit before their first heartbeat remains necessary.
   
   ### Steps to Reproduce
   
   Deterministic service-level sequence:
   
   1. Add a transaction subscription for a group and a topic with a valid broker-cluster route.
   2. Leave the group absent from `ProducerManager`, representing a pending first producer heartbeat.
   3. Run `scanProducerHeartBeat()`.
   4. Register the group, or change the mocked `groupOnline(group)` result to `true`.
   5. Run `scanProducerHeartBeat()` again, without adding another transaction subscription.
   
   The group is removed at step 3. Step 5 has no subscription from which to build a broker heartbeat.
   
   An end-to-end trigger is to send a transaction whose local result is `UNKNOWN` before the first producer heartbeat, keep the producer alive, and send no additional transaction messages. The failure depends on the scan falling between subscription creation and producer registration; it does not happen on every run.
   
   ### What Did You Expect to See?
   
   A recently created transaction subscription should survive a delayed first producer heartbeat and resume broker heartbeats once the producer becomes online.
   
   For the #5862 case, offline groups should receive no broker heartbeats, and subscriptions for producers that never register should eventually be reclaimed. Explicit group unregistration should still remove subscriptions immediately.
   
   ### What Did You See Instead?
   
   The subscription is removed immediately while the group is not yet registered. Registration alone does not restore it. The broker can then fail to find a channel for transaction checks, and the producer's transaction checker is not invoked.
   
   ### Proposed Fix
   
   Keep a monotonic last-activity timestamp on the existing `ClusterData` entries. Refresh it when subscriptions are added/replaced and while the group is observed online. When the group is offline, skip broker heartbeats immediately and expire subscriptions only after `channelExpiredTimeout` (currently 120 seconds by default). Preserve the existing explicit-unsubscribe path.
   
   Regression coverage should include delayed first registration, never-registered expiration, renewed subscription activity, online activity, replacement, and explicit unsubscription during the grace period.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Proxy removes transaction subscriptions before the first producer heartbeat [rocketmq]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #11170:
URL: https://github.com/apache/rocketmq/issues/11170#issuecomment-5711063752

   **Issue Evaluation**
   
   Category: `bug` | Status: **Confirmed**
   
   This is a valid bug in the Proxy transaction subscription management. The issue correctly identifies that `DefaultTransactionSubscriptionManager.updateSubscriptionData` unconditionally removes the existing topic subscription before registering the new one, which breaks the first producer heartbeat.
   
   **Root Cause:** The unconditional `removeTopicSubscription` in `updateSubscriptionData` clears the subscription before the new one is registered, causing the first heartbeat to fail.
   
   **Impact:** Transaction message producers experience subscription loss on first heartbeat, potentially causing message routing failures.
   
   **Severity:** Medium-High - affects transaction message reliability during producer initialization.
   
   **Assessment:** The proposed fix is appropriate:
   - Preserve the existing subscription until the new one is successfully registered
   - Ensure atomic update semantics
   
   This is related to the broader transaction message lifecycle management issues in Proxy.
   
   An automated fix proposal can be generated. Reply `/approve` to proceed with PR generation.
   
   ---
   *Automated evaluation by RockteMQ-AI*
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Proxy removes transaction subscriptions before the first producer heartbeat [rocketmq]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #11170:
URL: https://github.com/apache/rocketmq/issues/11170#issuecomment-5691867148

   **Issue Evaluation**
   
   Category: `type/bug` | Status: **Confirmed**
   
   The reported race condition in `ClusterTransactionService.scanProducerHeartBeat()` is valid. The issue correctly identifies that `ProducerManager.groupOnline(group)` returning false before the first producer heartbeat causes premature deletion of the transaction subscription via `groupClusterData.computeIfPresent()`.
   
   **Root Cause:** The scan does not distinguish between a group that has permanently gone offline and one that has not yet registered its first heartbeat. Once the subscription is removed, subsequent heartbeats only update `ProducerManager` but do not rebuild the deleted mapping.
   
   **Impact:** Transaction checks cannot reach the producer if no further transaction messages are sent after the race window. This affects reliability of transaction message processing in Proxy cluster mode.
   
   **Severity:** medium — The bug is deterministic under specific timing conditions but does not cause data loss; it prevents transaction status checks from reaching the producer.
   
   **Related:** This is a recurrence of #8316 (closed by stale bot) and relates to #5862/#5865. The earlier fix proposal #8320 introduced a grace period but was not merged.
   
   **Recommendation:** A grace period or subscription timestamp (as proposed in #8320) would prevent premature deletion. Alternatively, the scan could skip groups whose subscriptions were created within a recent time window.
   
   ---
   *Automated evaluation by github-manager*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Bug] Proxy removes transaction subscriptions before the first producer heartbeat [rocketmq]

["lizhimins (via GitHub)" <gi...@apache.org>]

lizhimins closed issue #11170: [Bug] Proxy removes transaction subscriptions before the first producer heartbeat
URL: https://github.com/apache/rocketmq/issues/11170


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org